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ABSTRACT 


Due  to  growing  operational  constraints  accelerated  by  the  Global  War  on  Terror, 
the  United  States  Navy  is  looking  for  alternative  methods  of  training  to  maintain  its  force 
in  a  high  status  of  readiness.  Updates  in  technology  over  the  last  decade  have  prompted 
Navy  officials  to  take  the  initiative  to  update  its  training  technologies.  Computer-Based 
Instruction  provides  alternative  means  of  training  so  that  the  training  of  war-fighters  can 
be  accomplished  efficiently  and  effectively,  saving  the  U.S.  Navy  time  and  resources 
while  maintaining  a  high  state  of  readiness.  The  goal  of  this  thesis  is  to  combine  the 
principles  of  Instructional  Design  Systems  (ISD)  technology  and  advanced  Simulation  in 
order  to  produce  a  multimedia  training  capability  for  Navy  and  Marine  Corps  Officers. 

Specifically  this  thesis  applies  ISD  principles  to  engineer  and  improve  the  Anti- 
Terrorism  Force  Protection  (AT/FP)  Officer  Course  Level  II  (ATO  Level  II  Course) 
lessons  for  use  by  Navy  and  Marine  Corps  officers.  The  SavageStudio  scenario 
generation  application,  originally  designed  for  assessment  of  force  protection  measures  in 
a  port  environment,  is  used  to  allow  students  to  develop  simulation  lessons  for  this 
course. 

The  final  product  of  this  thesis  is  a  training  course  consisting  of  two  lessons 
combining  Computer-  Based  Training  and  Simulation  providing  interactive  lessons  and 
exercises,  and  two  media  delivery  comparison  studies  with  results  for  the  Anti-Terrorism 
Force  Protection  lessons.  The  first  study  compared  media  effectiveness  of  Computer- 
Based  Training  versus  Classroom  instruction.  The  second  study  assessed  the 
effectiveness  of  Computer-Based  Training  and  the  use  of  simulation  for  AT/FP. 
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I.  INTRODUCTION 


A.  PROBLEM  STATEMENT 

The  United  States  has  been  under  terrorist  threat  and  attack  since  the  early  1980’s 
by  ideological,  political  and  religious  groups  that  do  not  agree  with  United  States  policies 
and  objectives  around  the  world.  According  to  sources  from  the  Center  for  Security 
Forces  (CENSEFOR  2006),  the  U.S.  military  has  suffered  more  casualties  due  to 
terrorism  than  in  combat  operations  during  the  period  since  the  Beirut  bombing  attack  in 
1983  to  the  beginning  of  Operation  Iraqi  Freedom.  Terrorist  groups  recognize  that  the 
strength  of  the  United  States  Armed  Forces  is  unparalleled  and  therefore,  it  is  impractical 
for  these  groups  to  challenge  the  U.S.  military  directly.  Instead  terrorist  groups  choose  to 
attack  vulnerabilities  of  U.S.  forces  in  order  to  gain  psychological,  ideological  and 
political  support. 

Events  such  as  the  USS  COLE  attack  on  12  October  2000,  while  refueling  in 
Yemen,  and  the  attempted  missile  attack  on  the  USS  ASHLAND  and  USS 
KEARSARGE,  while  moored  in  port  in  Jordan  on  19  August  2005,  reflect  the  Navy’s 
need  for  effective  Force  Protection  employment  of  assets  and  Anti-Terrorism  personnel 
training.  It  is  in  the  Navy’s  best  interest  that  all  hands  be  properly  trained  on  Anti- 
Terrorism  Force  Protection  (AT/FP)  tactics  techniques  and  procedures  (TTP)  to  deter, 
defend,  detect  and  mitigate  against  future  attacks. 

While  classroom  instruction  continues  to  be  the  most  preferred  delivery  method 
of  instruction,  there  has  been  a  significant  increase  in  technology  usage  over  the  past 
twenty  years  that  enables  other  delivery  methods  of  instruction.  Classroom  instruction  is 
a  cost-effective  tool  for  small  organizations  but  it  can  impose  several  constraints  for  large 
organizations  such  as  time,  cost  and  travel.  While  face-to-face  training  supports 
interaction  and  immediate  feedback,  learning  can  be  achieved  in  other  settings  where  the 
end  user  is  not  limited  by  time  or  location.  Users  can  access  the  instructional  lecture 
material  whenever  it  meets  their  needs. 
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A  large  organization  such  as  the  United  States  Navy  has  its  personnel  deployed  at 
locations  all  over  the  world,  accomplishing  a  variety  of  different  missions.  Training  costs 
are  expensive.  There  are  costs  to  maintain  the  classroom  facilities,  to  pay  the  instructors, 
to  maintain  the  teaching  materials  and  aids,  to  provide  travel  and  lodging  for  the  sailors, 
and  to  make  up  for  the  work  hours  lost  due  to  the  sailors  being  away  from  work. 

Properly  designed  Computer-based  Training  (CBT),  can  address  these  man  issues. 

B.  BACKGROUND  TRAINING  METHODOLOGIES 

As  a  result  of  the  Quadrennial  Defense  Review  of  1997,  the  Department  of 
Defense  initiated  the  Advanced  Distributed  Learning  (ADL)  effort.  The  intent  is  to  set 
forth  a  framework  to  provide  DoD  personnel  the  ability  to  achieve  high  quality  education 
and  training  tailored  to  individual  needs,  and  also  to  deliver  cost  effectively  whenever 
and  wherever  is  required  (Executive  Review  of  Navy  Training  Final  Report,  2001).  In 
2001  the  Executive  Review  Board  of  Naval  Training  concluded  that  traditional  classroom 
instruction  was  inefficient  relative  to  cost  and  total  time  to  train  each  student  (Executive 
Review  of  Navy  Training  Final  Report,  2001).  The  U.S.  Navy  commenced  a  review  of  its 
instructional  methodology  and  implemented  several  new  programs. 

One  such  new  program  was  implemented  at  the  Naval  Air  Technical  Training 
Center  in  Pensacola  Florida  and  the  Submarine  Learning  Center  in  Groton,  Connecticut. 
The  Apprentice  Technical  Training  Program  for  Electronic  Technical  Training  was 
created  under  the  Naval  Personnel  Development  Center  (NPDC).  Its  purpose  was  to 
review  and  assess  the  electronic  technician  training  curricula.  Extensive  reviews  of  the 
electronic  training  program  were  conducted  by  Subject  Matter  Experts  (SME)  as 
requested  by  the  Nida  Corporation,  which  was  contracted  to  restructure,  standardize  and 
manage  the  implementation  of  the  computer-based  learning  environment  for  the 
electronic  oriented  enlisted  rates.  The  computer-based  learning  environment  allowed 
students  to  proceed  from  basic  training  to  the  basic  electronic  school  without  having  to 
wait  for  the  class  to  be  full  enough  to  commence  a  new  class.  This  allowed  for  the  school 
administrators  to  schedule  more  classes,  more  often  with  fewer  students.  Students 
proceeded  with  lessons  at  their  own  pace  and  graduated  ahead  of  their  peers  when 
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compared  with  the  prior  classes.  Scenario-based  testing  was  conducted  and  students 
demonstrated  retention  of  proficiency  levels  in  measuring,  analyzing,  and  troubleshooting 
an  array  of  previously  studied  circuits  (Gulliver  2001).  The  Naval  Training  Facility 
located  in  Pensacola  Florida  was  closed  in  2004  during  Hurricane  Ivan  but  students  were 
able  to  access  the  courses  from  Great  Lakes  without  having  to  wait  for  the  installation  to 
fully  restore  all  its  classroom  facility  services. 

Another  relevant  Computer-Based  Training  (CBT)  /  Web-Based  Trainer  (WBT) 
training  program  was  implemented  in  2004  by  John  Ruffner,  Kathleen  Titley  and  Jim 
Fullbrook  titled,  Integrating  Technologies  for  Shipboard  Helicopter  Signaling  Skill 
Training.  A  prototype  CBT/WBT  of  the  Landing  Signal  Enlisted  (LSE)  course  created  by 
the  DCS  Corporation  Orlando,  Florida  was  tested  at  the  LSE  School  located  in  Naval  Air 
Station  (NAS)  Norfolk.  System  developers  and  Subject  Matter  Experts  (SMEs)  reviewed 
current  training  curricula  and  Personnel  Qualification  Standards  (PQS)  to  develop  the 
course  structure.  The  final  product  was  created  using  the  schoolhouse  curriculum  which 
matched  more  precisely  the  objectives  of  the  instruction.  Instructors  at  the  LSE  School 
expressed  strong  support  for  the  resulting  CBT.  They  commented  that  they  planned  to  use 
the  CBT  course  as  a  teaching  aid  to  supplement  classroom  instruction  and  discussion  as 
well  as  using  it  as  a  “study  ahead”  tool.  The  developers  are  working  to  formally  evaluate 
and  validate  the  CBT/WBT  and  also  are  working  on  a  virtual-reality  simulator,  so  that  it 
can  be  included  in  the  LSE  schoolhouse  curriculum  (Ruffner,  Titley  and  Fullbrook, 

2004). 

Updates  in  technology  over  the  last  decade  have  prompted  the  Chief  of  Naval 
Operations  (CNO)  to  take  the  initiative  to  update  training  technologies  for  the  United 
States  Navy,  as  seen  by  various  new  technologies  adopted  for  the  training  of  naval  forces. 
Of  particular  note  is  that  the  survival  of  our  sailors  depends  on  their  ability  to  be  properly 
trained  on  Anti-Terrorism  Force  Protection  (AT/FP)  use  and  employment  of  assets  to 
deter,  defend,  detect  and  mitigate  against  future  attacks.  The  creation  of  an  AT/FP  CBT 
with  embedded  simulation  can  improve  training  in  such  procedures.  Research  conducted 
by  Sitmann,  Kraiger,  Stewart  and  Wisher  (2006)  and  Hastings  and  Tracey  (2005)  points 
out  the  benefits  of  Computer-Based  Training  in  comparison  compared  to  other  more 
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traditional  methods  of  instruction.  CBT  and  simulations  can  be  effective  and  efficient, 
increasing  learning  and  knowledge  transfer  without  the  constraints  of  classroom 
instruction.  The  benefits  of  a  training  program  that  can  be  accessed  in  any  place  at  any 
time  gives  the  sailors  the  ability  to  grow  professionally  and  military. 

C.  OVERVIEW 

The  United  States  Navy  has  been  looking  at  new  technologies  to  provide  training 
to  its  sailors.  The  evolution  of  computers  and  web  applications  over  the  last  two  decades 
has  allowed  the  Navy  to  migrate  to  some  degree  prior  classroom  instruction  into  Web- 
based  Instruction  and  Web-based  portals  such  as  Navy  Knowledge  Online  (NKO).  These 
new  technologies  offer  some  advantages  that  are  necessary  in  today’s  tempo  of 
operations.  Currently  there  are  many  Navy  courses  that  can  be  restructured  to  allow  for 
the  delivery  of  the  material  using  this  Computer  and  Simulation-based  technologies. 

The  Mission  of  the  Naval  Education  and  Training  Command  (NETC)  is  to 
ensure  fleet  readiness  and  mission  accomplishment,  enhance  professional 
and  personal  growth,  and  enable  life-long  learning.  Guiding  principles 
include:  educate  and  train  fleet  requirements,  provide  the  tools,  learning 
and  guidance  that  enable  sailors  to  develop  to  their  fullest  potential, 
professionally  and  personally,  provide  responsive,  best  value  solutions 
through  rigorous  analysis  and  disciplined,  prioritized  alignment  with 
fleets,  measure  the  success  by  the  success  of  each  sailor  and  the  fleet, 
encourage  innovation  and  personal  initiative . (NETC  2007). 

As  operational  tempo  increases  based  on  the  demands  imposed  by  the  Global  War 
on  Terror  (GWOT),  the  United  States  Navy  needs  to  continue  investigating  and  applying 
technologies  that  allow  sailors  to  be  trained  to  their  fullest  potential  with  minimum 
impact  to  the  Navy’s  mission. 

D.  MOTIVATION 

As  a  former  Training  Officer,  this  author  has  experienced  first  hand  the  rigorous 
training  demands  imposed  on  afloat  and  ashore  commands  from  the  increased  operational 
tempo  of  the  GWOT.  Commands  are  required  to  achieve  and  maintain  higher  levels  of 
readiness  in  all  warfare  areas;  however  the  operational  day  remains  twenty  four  hours 
long,  with  the  same  number  of  sailors  and  command  budgets  steadily  decreasing. 
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In  today’s  Navy  sailors  no  longer  specialize  in  a  single  area.  They  must  be  trained 
to  master  multiple  warfare  areas  such  as  Damage  Control  (DC),  Visit  Board  Search  and 
Seizure  (VBSS)  and  Force  Protection  (AT/FP)  in  addition  to  their  primary  in-rate  duties. 

It  has  been  difficult  for  commands  to  decide  when  to  allow  sailors  time  away 
from  the  command  to  attend  courses  in  a  classroom  setting,  because  the  sailor  may  be 
needed  for  other  ship  operations  allowing  the  sailor  to  be  absent  may  decrease  the 
command’s  state  of  readiness.  Decreasing  budgets  are  also  a  factor  when  making  these 
decisions  because  the  command  may  not  have  the  funds  needed  to  send  all  sailors ’s  on 
Temporary  Additional  Duty  (TAD)  to  a  school  for  the  required  training. 

Computer-based  Training  (CBT)  and  Simulation-based  Training  (SBT)  are 
underutilized  technologies  that  can  help  in  the  training  field.  Modeling  and  Simulation 
technologies  can  be  used  to  create  Scenarios  that  exercise  the  individual  and  group 
learning  skills.  There  have  been  previous  attempts  to  use  CBT  and  SBT  to  train  sailors 
onboard  ships.  Some  have  been  somewhat  successful,  such  as  Division  Officer  Training 
at  Sea  from  the  Surface  Warfare  Officer  School  (SWOS),  Newport  Rhode  Island.  Other 
training  attempts  have  been  less  successful,  such  as  the  Navy’s  General  Military  Training 
(GMT).  Often  the  less-than-successful  results  are  not  due  to  the  technology  used  to 
deliver  the  instruction,  rather  the  way  that  the  courses  are  organized  and  developed.  If  the 
material  and  content  of  a  course  is  poorly  developed,  instruction  results  will  be  poor  no 
matter  what  technology  is  used.  The  Instructional  System  Design  (ISD)  approach  can  be 
used  to  create,  implement  and  evaluate  course  materials  that  can  be  later  paired  to  the 
available  technologies  to  achieve  maximum  knowledge  gains. 

E.  OBJECTIVES 

The  objective  of  this  thesis  research  is  to  develop  and  test  a  prototype  instruction 
system  to  teach  Navy,  Marine  Corps  Officers  and  Senior  Enlisted  in  AT/FP  using  the 
Instructional  Systems  Design  (ISD)  approach.  The  thesis  tasks  that  need  to  be 
accomplished  in  order  to  meet  this  thesis  purpose  are: 
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1.  Conduct  a  study  into  the  Anti-Terrorism  Officer  (ATO)  Level  II  course  to 
analyze  what  training  objectives  are  achievable,  based  on  the  material  that  needs  to  be 
taught  and  the  learner’s  current  capabilities. 

2.  Conduct  a  task  analysis  based  on  cognitive  and  skill  tasks  needed  to  define  the 
learning  objectives  for  the  training  material,  based  on  the  information  gathered  from 
SME’s  and  current  course  notes. 

3.  Develop  a  training  delivery  format,  instructional  methods  and  course  content 
organization. 

4.  Review  and  evaluate  different  media  technologies  that  allow  the  delivery  of  the 
course  content,  taking  into  consideration  time,  resources,  budget  constraints  and  course- 
learning  goals. 

5.  Conduct  experiments  that  focus  on  determining  the  most  effective  training 
strategy. 

6.  Report  results  and  recommendations  to  Navy  training  commands  in  order  to 
encourage  further  development  and  deployment  of  these  important  new  tactical  decision 
aids  and  technologies. 

F.  THESIS  ORGANIZATION 

The  thesis  is  organized  as  follows.  Chapter  II  reviews  background  technologies, 
the  Instructional  System  Design  and  previous  work  that  motivated  this  thesis.  Chapter  III 
provides  and  in-depth  look  at  the  Analysis,  Design  and  Development  phases  for  the 
creation  of  the  CBT  lessons.  Chapter  IV  provides  a  detailed  explanation  of  the 
methodology  used  in  the  implementation  phase  of  the  thesis  project.  Chapter  V  outlines  a 
step-by-step  instruction  on  how  to  use  SavageStudio  in  the  creation  of  a  Force  Protection 
Simulation  Scenario.  Chapter  VI  results  of  the  two  studies  conducted  are  discussed  in 
great  detail.  Chapter  VII  provides  a  summary  of  the  conclusions  from  this  work  and 
provides  recommendations  for  future  work.  Appendix  A  contains  the  task  analysis  for 
lesson  development.  Appendix  contains  lesson  enabling  objectives.  Appendix  C  contains 
media  selection  criteria.  Appendix  D  and  Appendix  E  contain  lesson  storyboards. 
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Appendix  F  thru  Appendix  I  contain  lesson  pre  and  post-tests.  Appendix  J  contains 
sample  port  brief  information  sheet.  Appendix  K  contain  Institution  Review  Board  (IRB) 
forms  sample.  Appendix  L  contains  the  student  questionnaire.  Appendix  M  contains  the 
SavageStudio  exercise  task  checklist.  Appendix  N  contains  an  Analyst  report  sample. 
Appendix  O  contains  supporting  materials  and  information  on  how  to  request 
SavageStudio  and  AT/FP  lesson  DVDs. 
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II.  RELATED  WORK 


A.  INTRODUCTION 

This  chapter  reviews  prior  Naval  Postgraduate  School  (NPS)  theses  that  served  to 
motivate  this  thesis  work.  It  also  discusses  the  Anti-Terrorism  Officer  Level  II  course 
(ATO  Level  II  course),  the  Scenario  Authoring  and  Visualization  for  Advanced 
Graphical  Environments  (Savage)  Studio  simulation  environment,  and  the  Instructional 
Systems  Design  (ISD)  concept  used  in  the  completion  of  this  thesis.  Relevant  research 
about  training  effectiveness  between  Computer-Based  Training  (CBT)  and  Classroom 
Instruction  (Cl)  is  also  discussed.  Additional  infonnation  on  the  topics  related  to  this 
thesis  can  be  found  in  the  references  section  at  the  conclusion  of  this  thesis. 

B.  ANTI-TERRORISM  OFFICER  LEVEL  II  COURSE  (ATO  LEVEL  II 

COURSE) 

The  purpose  of  the  Anti-Terrorism  Officer  (ATO)  course  is  to  incorporate  the 
Level  II  mandated  training  requirements  contained  in  DODINST  2000.16  to  provide  the 
ATO  with  the  necessary  skills  and  knowledge  of  the  six  elements  of  the  AT  program: 
Planning,  Risk  Management,  Training,  Exercises,  Resources  and  AT  Program  Review. 
Graduates  are  qualified  to  manage  their  command's  anti-terrorism  program  ashore  and 
afloat  (CENSECFOR  2006).  Lesson  topics  include  the  concept  of  defense  in  depth, 
development  of  pre-planned  responses,  duties  and  responsibilities  of  an  ATO,  use  of 
force,  introduction  to  terrorism  and  terrorism  operations,  AT  resources  and  tools,  tactic 
techniques  and  procedures  (TTP)  and  AT  plan  development  and  procedures 
(CENSECFOR  2006).  Training  is  accomplished  through  a  combination  of  classroom 
lectures  and  practical  scenarios  relevant  to  both  shore-based  and  afloat  commands. 

The  scope  of  the  course  is  to  train  Navy  commissioned  officers  and  selected 
enlisted  E-7  and  above,  USMC  personnel  and  DoD  equivalent  civilians  as  Anti- 
Terrorism  Officers  (ATO).  The  course  provides  instruction  in  the  U.S.  Navy  and  Marine 
Corps  Anti-Terrorism  and  Force  Protection  (AT/FP)  programs  as  described  in 
OPNAVINST  3300.55,  MCO  3302. 1C  and  DODINST  2000.16  (CENSECFOR  2006). 
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Course  material  includes  the  process  for  developing  of  comprehensive  anti-terrorism 
plans,  AT  program  management  and  AT  program  assessments  and  execution.  The  ATO 
Level  II  course  is  organized  by  classroom  instruction  followed  by  a  practical  scenario 
development.  The  course  consists  of  five  days  of  instructor  led  course  instruction.  The 
instructor  uses  presentation  slides,  class  discussion  and  an  assortment  of  visual  aids 
during  the  initial  three  days  of  the  course. 

Topics  covered  in  the  ATO  Level  II  Training  Course  are  presented  in  Figure  1. 


Unit  1 

1-1-1  -  Introduction 

1-2-1  -  Introduction  to  Terrorism/Terrorist  Operations 

1-3-1  -  Introduction  to  Terrorism  Program 

1-4-1  -  (FOUO)  Improvised  Explosive  Devices  (IED's) 

1-5-1  -  Use  of  Force/Deadly  Force 

1- 6-1  -  Anti-terrorism  Officer  Infonnation  and  Tools 
Unit  2 

2- 1-1  -  Defense  in  Depth/Preplanned  Responses 

2-1-2  -  Pre-Planned  Responses 

2-2-1  -  Force  Protection  Conditions  (FPCON's) 

2-3-1  -  Surveillance 

2- 4-1  -  Individual  Protective  Measures 
Unit  3 

3- 1-1  -  Anti-terrorism  Risk  Management 

3-2-1  -  Anti-terrorism  Threat  Assessment  and  Threat  Levels 
Unit  4 
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4-1-1  -  Introduction  to  Anti-terrorism  Plans 

4-1-2  -  Sample  In-port  Security  Plan 

4-2- 1  -  Anti-terrorism  Planning  Process 

4-3-1  -  Anti-terrorism  Plan  Format 

4-3-2  -  JAT  Guide  Anti-terrorism  Plan  Basic  Plan 

4-4- 1  -  AT  Program  Funding 

4-5-1  -USS  Cole  Case  Study 

4-6-1  -  Khobar  Towers  Case  Study 

4-7-1  -  Task  One  -  Force  Protection  Conditions  Exercises 

4-7-2  -  Post  Orders  for  Main  Gate  and  Industrial 

4-8-1  -  In-port  Security  Plan  (ISP)  Practical  Application 

4-9-1  -  Ashore  Table  Top  Exercise  Brief 

4-9-2  -  Afloat  Table  Top  Exercise  Brief 

Figure  1.  The  ATO  Level  II  Training  Course  Outline  includes  a  variety  of  AT/FP 

topics. 

On  the  fourth  day  students  are  divided  into  teams  to  develop  a  force-protection 
plan  for  a  given  scenario.  On  the  fifth  day  the  student-prepared  plans  are  presented  by 
each  team  leader.  Feedback  is  provided  by  the  instructors  and  other  students. 

C.  RELEVANT  NAVAL  POSTGRADUATE  SCHOOL  THESES 

LT.  James  Harney’s  thesis,  Analyzing  Anti-Terrorist  Tactical  Effectiveness  of 
Picket  Boats  for  Force  Protection  of  Navy  Ships  Using  X3D  Graphics  and  Agent  Based 
Simulation  (March  2003)  describes  the  use  of  open  source,  web,  modeling  and  simulation 
technologies  used  in  the  development  of  a  prototype  to  aid  tactical  operators  in  the  Anti- 
Terrorism  and  Force  Protection  planning.  The  prototype  allows  for  the  operator  to  create, 
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visualize  and  analyze  Anti-Terrorism  and  Force  Protection  “what-if  ’  scenarios.  The 
operators  use  these  hypothetical  scenarios  to  aid  in  the  development  of  an  effective  Force 
Protection  plan. 

LT  Patrick  Sullivan’s  thesis,  Evaluating  the  Effectiveness  of  Waterside  Security 
Alternatives  for  Force  Protection  of  Navy  Ships  and  Installations  Using  X3D  Graphics 
and  Agen-Based  Simulation  (September  2006)  expands  LT  Harney’s  thesis  to  include 
other  force  protection  measures  for  Navy  ships  and  installations.  The  prototype  created 
by  LT  Harney  was  refined  during  LT  Sullivan’s  thesis  to  provide  Naval  Installation 
Security  Planners,  Harbor  Operations  Support  Staff  and  shipboard  Force  Protection 
Officers  the  ability  to  evaluate  security  alternatives  using  simulations.  Furthermore  the 
Scenario  Authoring  and  Visualization  for  Advanced  Graphical  Environments 
(SavageStudio)  scenario  tool  provides  Naval  Installation  Security  Planners  with  the 
ability  to  model,  simulate  and  analyze  new  systems  being  considered  in  order  to  assess 
overall  risk  reduction  and  return  on  investment. 

D.  SCENARIO  AUTHORING  AND  VISUALIZATION  FOR  ADVANCED 

GRAPHICAL  ENVIRONMENTS  (SAVAGE)  STUDIO 

SavageStudio  is  a  scenario-generation  analysis  and  visualization  tool  that 
provides  the  user  with  the  ability  to  create  complete  AT/FP  simulations  with  minimal 
setup  and  design  (Sullivan  2006).  SavageStudio  allows  users  to  fully  generate  scenarios 
that  can  be  played  back  and  analyzed  using  discrete  event  simulation  (DES)  and 
Extensible  3D  (X3D)  graphics.  The  scenario  generation  is  accomplished  by  leveraging 
the  extensible  markup  language  (XML)  architecture  of  the  Savage,  X3D  model  archive 
and  Viskit  discrete  event  simulation  to  generate  complex  simulations  for  visualization 
and  analysis. 

The  SavageStudio  scenario  generation  tool  provides  the  user  with  scenario 
creation  capabilities  using  the  same  type  of  drag-and-drop  interfaces  used  in  most  Navy 
Information  Technology  for  the  21st  century  (IT-21)  applications.  The  interface  allows 
users  to  select  models  (locations,  friendly  assets,  hostile  assets,  neutral  assets,  etc)  that 
will  be  used  in  the  scenario.  During  scenario  configuration,  the  user  can  change  physical 
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parameter  values  (maximum  speed,  cruising  speed,  vessel  draft,  range,  altitude,  etc.)  for 
each  model  and  define  their  behaviors  (friendly  patrol  craft,  hostile  explosive-laden 
vessel,  etc).  Once  the  models  and  their  behaviors  have  been  selected  and  assigned, 
SavageStudio  creates  a  Viskit  assembly  file  and  an  X3D  model  of  the  scenario  which 
corresponds  to  the  assembly  file. 

Viskit  is  a  visual  programming  tool  for  modeling  behaviors  and  tactics  that  allows 
designers  to  use  the  Event  Graph  (EG)  methodology  to  represent  processes  and 
mathematical  equations  as  discrete  event  models  (DES).  A  sample  event  graph  is 
provided  in  Figure  2.  Assembling  multiple  event  graphs  into  a  single  simulation  allows 
the  user  to  construct  tactical  scenarios  of  intent  and  derive  numerical  analysis  from  their 
results.  In  DES  the  simulation  time  clock  advances  as  events  occur  rather  than  running  in 
constant  blocks  of  time.  This  approach  enables  faster  running  times,  more  scenario 
replications,  and  more  thorough  analysis  results. 

The  Viskit  Graphical  User  Interface  (GUI)  written  in  Java  is  used  to  implement 
the  event  graph  methodology.  The  Simkit  GUI  allows  for  statistical  analysis  of  a 
simulation.  The  Diskit,  Java  API  (application  programming  interface)  extends  the 
SIMKIT  libraries.  The  DISKIT  libraries  enable  the  use  of  the  IEEE  Distributed 
Interactive  Simulation  Protocol  (DIS)  allowing  the  animation  of  3D  objects  in  a  virtual 
environments  and  communications  between  different  simulations.  The  Viskit,  GUI 
allows  the  user  with  event  graph  knowledge  to  create  computer  simulation  leveraging 
Simkit  and  Diskit  libraries.  It  is  important  to  recognize  that  the  user  does  not  need  to 
know  how  to  write  Java  code,  he  just  needs  to  know  how  create  an  event  graph.  The 
event  graph  produced  using  the  Viskit  GUI  is  then  saved  as  XML  documents.  This  XML 
documents  are  transformed  into  JAVA  source  code  using  JAXB  implementing  the 
SIMIKT  and  DISKIT  libraries. 
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Figure  2.  The  above  figure  is  an  example  of  the  run  method  in  event  graph  notation 

(from  Buss  2004) 

The  3D  model  of  the  scenario  which  corresponds  to  the  assembly  file  created  by 
SavageStudio  is  created  in  Extensible  3D  Graphics  X3D  Standard  (Brutzman,  Daly 
2007).  X3D  graphics  is  an  ISO  standard  used  in  web  pages,  database  applications  and 
visual  simulations.  For  more  information  regarding  these  topics  refer  to  LT.  Sullivan’s 
thesis  (Sullivan  2006). 

E.  INSTRUCTIONAL  SYSTEM  DESIGN 

The  Instructional  System  Design  (ISD)  approach  is  the  guiding  methodology  used 
in  the  lesson  development  for  this  study.  The  goal  of  the  ISD  approach  is  to  select  and 
use  the  correct  instructional  system  to  deliver  the  required  knowledge  to  the  learner.  The 
development  of  the  instructional  system  is  structured  in  3  parts:  input,  process  and  output. 
(Ford  1996) 

The  input  portion  is  for  the  ISD  is  intended  to  help  understand  the  differences  in 
the  learner’s  experience  levels,  the  learner’s  current  level  of  knowledge  about  the  subject, 
and  the  motivations  of  the  learner.  This  information  can  be  summarized  as  the  learner’s 
background  coupled  with  the  desired  goal  of  the  instruction.  The  process  portion  is  for 
the  ISD  develops  the  course  and  the  media  that  delivers  the  instruction  to  the  learner.  The 
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output  portion  for  the  ISD  is  used  to  evaluate  the  effectiveness  of  the  training  to  ensure 
that  the  system  meets  the  training  goals.  Figure  3  contains  the  ISD  concept  map  by 
Donald  Clark. 


Figure  3.  Instructional  System  Design  (ISD)  design  concept  map  by  Donald  Clark 
http://www.nwlink.com/~donclark/hrd/sat2.html  (1995). 

There  are  more  than  100  different  ISD  models  (Kruse  2007)  but  almost  all  are 
based  on  the  “ADDIE”  model.  The  ADDIE  model,  which  stands  for  Analysis,  Design, 
Development,  Implementation  and  Evaluation,  is  used  by  the  training  community  to 
standardize  instruction  and  to  impose  consistent  quality  standards  (Ciavarelli  2007).  An 
ADDIE  model  as  described  by  Steven  J.  McGriff  is  depicted  in  Figure  4.  A  major  benefit 
of  using  the  ADDIE  model  is  that  each  step’s  outcome  feeds  the  subsequent  steps, 
ensuring  that  developers  don’t  proceed  to  subsequent  steps  without  first  completing  the 
previous  necessary  step. 
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The  ADDIE  Model 
is  an  iterative 
instructional  design 
process,  where  the 
results  of  the 
formative  evaluation 
of  each  phase  may 
lead  the  instructional 
designer  back  to  any 
previous  phase 

The  end  product  of 
one  phase  is  the 
starling  product  of 
the  next  phase. 


Figure  4.  ADDIE  model  as  described  by  Steven  J.  McGriff,  Instructional  Systems, 

College  of  Education,  Penn  State  University  (2000). 


The  ADDIE  model  steps  are  briefly  summarized  in  the  following  paragraphs. 

In  the  Analysis  phase  the  problem  needs  to  be  defined  using  a  job  task  analysis, 
determining  what  training  is  needed  and  why  it  is  important.  Developers  identify  the 
possible  solutions  to  the  problem,  using  task  analysis,  developers  can  explicitly  define 
roles,  tasks  and  outcomes  of  the  targeted  training.  Developers  define  a  sequence  of  tasks 
coupled  with  the  required  equipment  that  facilitate  the  execution  of  the  task  and  measures 
of  job  perfonnance  are  provided.  The  target  audience  has  to  be  identified,  developers 
need  to  analyze  their  knowledge  and  skills  in  order  to  detennine  what  they  need  to 
complete  the  training  and  achieve  satisfactory  perfonnance.  The  outputs  of  this  phase  are 
the  inputs  for  the  Design  phase. 

The  Design  phase  uses  the  outputs  from  the  Analysis  phase  to  plan  a  strategy  and 
build  a  blueprint  for  the  training  program.  First  the  scope  of  the  project  is  defined;  writing 
objectives  course  content  test  and  evaluation,  delivery  system  and  instructional  strategies 
and  sequence  of  instruction  and  resources. 

The  Development  phase  uses  the  outputs  of  the  design  phase  to  generate  lesson 
plans,  student  and  teacher  guides  and  the  instructional  materials  required  to  complete  the 
lessons.  Here  the  actual  training  program  is  constructed  using  courseware  authoring  and 
software  as  specified  in  the  design  (Ciavarelli  2007). 
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The  Implementation  phase  has  two  parts.  The  pilot  phase  creates  or  designs  a  pilot 
study  and  then  delivers  the  instruction  to  a  small  group  of  subjects  to  test  the  efficacy  of 
the  training  instruction.  After  the  pilot  study  is  completed,  the  second  part  is  to  revise  the 
prototype  and  then  deliver  the  overall  training  solution  to  the  end-user  population. 

The  Evaluation  phase  measures  the  overall  effectiveness  of  the  training.  Even 
though  the  evaluation  phase  is  the  last  phase  of  the  ADDIE  model,  developers  are 
constantly  evaluating  the  system  to  correct  or  improve  the  instruction  before  the  final 
implementation.  Thus  evaluation  takes  place  in  all  the  phases  of  ISD  (Ciavarelli  2007). 
The  final  version  of  evaluation  determines  whether  the  training  program  achieves  the 
desired  results  and  meets  the  proposed  objectives. 

F.  COMPUTER-BASED  TRAINING  (CBT)  VERSUS  CLASSROOM 

INSTRUCTION  (Cl)  EFFECTIVENESS 

The  literature  on  CBT  effectiveness  reveals  mixed  results.  Individual  studies 
provide  examples  of  CBT  performing  better,  worse  or  having  no  effect.  A  1992  IBM 
study  demonstrated  an  up  to  70  percent  decrease  in  training  time,  with  learning  gains 
over  50  percent  (Whitehouse  &  Pellegrin,  1995).  Coppola  and  Myre  (2002)  addressed 
several  previous  studies  that  claimed  CBT  was  not  as  effective  as  Classroom  Instruction 
(Cl).  In  Wilson,  Majsterek  and  Simmons  (1996)  four  elementary  students  with  learning 
disabilities  perfonned  worse  with  ‘Mathblaster’  CBT  than  with  flashcards  in  a  Cl  setting. 
In  Lee  (1994)  twenty  six  teachers  did  not  learn  well  when  they  where  required  to  use  a 
CBT  program  to  teach  BASIC.  Coppola  and  Myre  cited  those  studies  as  examples  to 
demonstrate  alternate  hypotheses  that  must  be  controlled  for  such  as  computer  skills  and 
attitude  towards  the  material. 

When  multiple  studies  are  evaluated  together,  those  cases  that  show  significant 
differences  in  performance  between  media  appear  to  be  outliers.  They  can  be  explained 
by  alternate  hypotheses,  or  used  evaluation  samples  that  are  not  representative  of  the 
target  population.  Chumley- Jones,  Dobbie  and  Alford  (2002)  reviewed  76  articles  from 
1966  to  2002  about  medical,  dental  and  nursing  CBT.  They  observed  that  studies  with 
control  groups  where  the  educational  content  was  constant  demonstrated  no  difference  in 
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knowledge  gains.  A  meta-analysis  of  96  studies  from  1996  to  2005  reached  a  similar 
conclusion.  Sitmann,  Kraiger,  Stewart  and  Wisher  (2006)  evaluated  studies  of  adults  18 
years  or  older  comparing  WBL  and  Cl  for  job-related  training  programs.  Training  times 
ranged  from  one  to  120  days.  Only  studies  that  provided  enough  data  to  allow  for  the 
calculation  of  a  d  statistic,  or  where  the  author  provided  such  data  when  requested,  were 
used.  The  various  aspects  of  each  study,  such  as  response  time,  same  or  different  method, 
experiment  or  case  study  etc.,  were  then  converted  to  a  nonnalized  scale  by  two  raters 
with  a  third  rater  breaking  any  ties.  They  then  looked  at  a  subset  of  the  studies  that  used  a 
true  experimental  design,  a  subset  that  used  equal  instructional  methods,  and  a  subset  that 
looked  at  using  CBT  in  addition  to  CI.  They  further  separated  instruction  of  declarative 
or  procedural  knowledge  within  each  of  the  main  subsets.  Results  are  summarized  as 
follows. 

The  overall  results  indicated  WBI  was  six  percent  more  effective  then  CI 
for  teaching  declarative  knowledge,  the  two  delivery  media  were  equally 
effective  for  teaching  procedural  knowledge.  However,  WBI  and  CI  were 
equally  effective  for  teaching  declarative  knowledge  when  the  same 
instructional  methods  were  used  to  deliver  both  WBI  and  CI,  suggesting 
media  effects  are  spurious  and  supporting  Clarks  (1983,  1994)  theory  (p. 

623). 

Clark’s  theory  is  best  demonstrated  by  his  often-cited  truck  analogy.  “The  best 
current  evidence  is  that  media  are  mere  vehicles  that  deliver  instruction  but  do  not 
influence  student  achievement  any  more  than  the  truck  that  delivers  our  groceries  causes 
changes  in  our  nutrition  (Clark,  1983,  p.  445).” 

Hastings  and  Tracey  (2005)  evaluate  the  articles  and  counter  articles  written  by 
Clark  in  1983  and  1994  and  Kozma  in  1991  and  1994.  “His  [Clark’s]  position  has  always 
been  that  any  given  method  can  be  developed  for  delivery  by  more  than  one  medium.  In 
1983  and  even  1994,  Clark’s  argument  was  valid.  In  2005  it  is  valid  for  many,  but  not 
all,  instructional  methods  and  delivery  medium  pairings.  We  assert  that  today,  computers 
have  unique,  non  replicable  capabilities  and  therefore  can  support  instructional  methods 
that  other  media  cannot”  (p.  29).  Hastings  and  Tracey  purport  advances  in  computers  and 
the  Internet  over  the  last  22  years  have  transformed  Clark’s  computer-as-a-truck  into  a 

supersonic  jet.  As  an  example,  in  1991  Kozma  mentions  that  due  to  size  and  weight  one 
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is  more  likely  to  leam  from  a  book  than  a  computer  while  riding  a  bus,  but  that  may 
change  with  advances  in  technology.  Kozma  asserts  computers  have  unique  qualities  that 
if  utilized  properly,  can  make  CBT  more  effective  than  CL 

Greitzer  (2002)  describes  how  content  developed  for  CBT  was  adapted  for  Cl 
instruction,  thus  improving  both.  The  results  reported  by  Sitzmann  et  al.  (2006)  and 
Schlager  (1994)  are  directly  applicable  to  the  U.S.  Navy  because  they  only  looked  at  true 
experiments  on  adults  with  at  least  basic  computer  and  learning  skills.  There  are  many 
case  studies,  quasi-experiments  and  experiments  to  support  claiming  an  advantage  of 
CBT  or  CL  When  one  looks  only  at  studies  in  which  true  experimental  methods  were 
used,  comparing  the  same  material  given  to  subjects  similar  to  the  population  of  the  U.S. 
Navy,  no  difference  was  found  between  CBT  and  CI. 

G.  SUMMARY 

This  chapter  reviews  prior  Naval  Postgraduate  School  (NPS)  theses  that  served  to 
motivate  this  thesis  work.  It  also  discusses  the  Anti-Terrorism  Officer  Level  II  course 
(ATO  Level  II  course),  the  Scenario  Authoring  and  Visualization  for  Advanced 
Graphical  Environments  (Savage)  Studio  simulation  environment  and  the  Instructional 
Systems  Design  (ISD)  concept  used  in  the  completion  of  this  thesis.  Relevant  research 
about  training  effectiveness  between  Computer-Based  Training  (CBT)  and  Classroom 
Instruction  (CI)  is  discussed.  Additional  information  on  the  topics  related  to  this  thesis 
can  be  found  in  the  references  section  at  the  conclusion  of  this  thesis.  Further  information 
and  greater  in-depth  knowledge  of  the  subjects’  is  provided  in  the  list  of  references. 
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III.  ANALYSIS,  DESIGN  AND  DEVELOPMENT 


A.  INTRODUCTION 

This  chapter  provides  an  in-depth  look  at  the  analysis,  design  and  development  of 
the  lessons  implemented  using  the  CBT  and  SavageStudio  scenario  generation  tool.  The 
use  of  the  instructional  design  model  helps  to  ensure  that  the  instruction  developer  makes 
important  choices  about  the  course  structure  and  methods,  thus  ensuring  that  the  training 
objectives  are  achieved. 

B.  ANALYSIS 

1.  Needs  Analysis 

With  the  revolution  in  Training  (Executive  Review  of  Navy  Training  2001)  the 
United  States  Navy  has  been  looking  at  new  or  alternative  technologies  to  provide 
training  to  its  sailors.  The  U.S.  Navy  is  restructuring  some  of  its  courses  to  allow  for  the 
delivery  of  the  material  using  Computer  and  Simulation-based  technologies.  During  the 
last  decade,  the  terrorist  threat  has  been  increasing  as  evidenced  by  terrorist  attacks 
against  U.S.  forces  around  the  world.  There  is  a  great  need  to  train  Naval  and  Marine 
Corps  Officers  and  Senior  Enlisted  in  the  concepts  of  AT/FP.  As  operational  tempo 
imposed  by  the  GWOT  increases,  instructors  need  to  look  to  other  to  look  into 
technologies  that  allow  sailors  to  be  trained  to  their  fullest  potential  anywhere  at  any 
time. 

Clearly  the  Navy’s  needs  are: 

1 .  Efficient  or  effective  technologies  that  encourage  faster  learning  and  efficiency 
implementing  the  concepts. 

2.  Training  delivery  both  ashore  and  afloat  using  established  infrastructure. 

2.  Task  Analysis 

Formal  task  analysis  helps  the  developer  determine  what  specific  tasks  need  to  be 
accomplished  in  order  to  fulfill  each  learning  objective.  Tasks  are  broken  down  in 
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subtasks,  and  each  subtask  is  matched  to  a  learning  objective.  The  task  analysis  also  helps 
developers  build  perfonnance  measures  that  help  to  determine  student  success  or  failure 
when  performing  the  task.  A  detailed  task  analysis  supporting  the  objectives  of  this  thesis 
is  provided  in  Appendix  A. 

3.  Audience 

A  thorough  understanding  of  the  participants  is  essential  for  developing 
successful  instructional  material.  There  is  a  need  to  develop  an  audience  profile  to  ensure 
that  the  instructional  design  is  compatible  with  the  students’  skill  and  knowledge  level. 
The  lessons  in  this  course  are  intended  for  Officers  and  Senior  Enlisted  assigned  to 
AT/FP  duties.  The  group  of  Officers  is  comprised  of  all  Naval  Officers,  all  Marine  Corps 
Officers,  Limited  Duty  Officers  in  the  military  police  rates  and  Senior  Enlisted  in  the  Fire 
Control,  Gunners  mates  and  Master  at  Arms  rates.  Given  the  nature  of  the  military  rank 
structure,  most  Limited  Duty  Officers  and  Senior  Enlisted  in  the  mentioned  rates  often 
possess  some  post-secondary  college  education  or  degree.  This  indicates  that  Officers  and 
Senior  Enlisted  continued  their  education  during  the  course  of  their  military  service. 

Commissioned  Officers  are  required  to  have  a  college  degree.  With  the 
implementation  of  IT-21  and  the  Navy  Marine  Corps  Internet  (NMCI)  and  the 
educational  background  of  the  audience,  it  can  be  assume  that  they  have  at  least  basic 
computing  skills  necessary  for  the  applications  used  in  this  research.  The  level  of 
enthusiasm  is  high  among  the  students  selected  to  attend  the  course,  and  they  are  usually 
selected  to  attend  the  course  due  to  high  level  of  confidence  by  their  superiors. 

The  background  knowledge  that  the  students  have  in  the  Force  Protection  subject 
varies,  from  basic  to  advanced.  Some  Naval  Officers  ranks  0-1  thru  0-3  have  basic 
knowledge,  while  the  Limited  Duty  Officers  and  the  Senior  Enlisted  have  more  advanced 
knowledge  and  have  been  exposed  to  the  subject  since  some  have  served  in  the  AT/FP 
rates. 
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C.  DESIGN 

1.  Learning  Objectives 

The  learning  objectives  are  the  center  of  each  lesson  design  and  help  describe  the 
knowledge,  skills  and  attitudes  (KSAs)  expected  of  the  student  after  completion  of  the 
lesson  activities.  By  meeting  the  learning  objectives  the  student  will  be  able  to  achieve 
the  course  goal.  The  objectives  clearly  state  what  the  student  is  expected  to  know 
(knowledge)  or  perform  (skill)  at  the  end  of  each  lesson,  and  to  what  degree  in  order  to 
properly  demonstrate  successful  completion  of  the  objective. 

The  overall  student  learning  objectives  for  the  ATO  course  are  defined  as: 

1 .  Manage  the  overall  Anti-terrorism  Program  as  the  Commanding 
Officer’s  primary  advisor  for  a  shore-based  or  afloat  command,  including  personnel, 
physical  resources,  and  training  requirements. 

2.  Develop  and  implement  Pre-Planned  Responses  that  will  be  effective 
mechanisms  to  detect,  deter,  defend  and  mitigate  likely  terrorist  threats  against  U.S.  Navy 
assets  and  installations. 

3.  Assess  the  anti-terrorism  environment,  situation  and  unit  readiness  for 
an  ashore  or  afloat  command,  given  current  contingencies  and  sources  of  information. 

4.  Develop  a  comprehensive  Anti-terrorism  Plan  incorporating  anti¬ 
terrorism  principles,  concepts,  program  elements  and  standards  across  the  range  of 
military  operations,  at  any  echelon  of  command  both  afloat  and  ashore,  utilizing  the  AT 
planning  process. 

The  learning  objectives  for  the  created  lesson  are  defined  as: 

1 .  Define  the  importance  of  the  Defense  in  Depth  concept  in  accordance 
with  Navy  AT/FP  Tactics,  Techniques  and  Procedures  (TTP). 

2.  Apply  the  procedures  of  the  Defense  in  Depth  concept  to  a  ship’s  in¬ 
port  preparations. 
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3.  Assess  the  anti-terrorism  environment  and  situation  to  identify  port 

vulnerabilities. 

4.  Define  and  assess  the  anti-terrorism  environment  and  situation  to 
identify  critical  assets  and  vulnerabilities  for  a  ship  in-port. 

5.  Assess  the  anti-terrorism  environment  and  situation  to  develop  a  plan 
that  mitigates  vulnerabilities  to  an  acceptable  risk  for  a  ship  in-port. 

6.  Assess  the  anti-terrorism  environment  and  situation  to  assess  risk  to  a 
ship  in-port  and  develop  appropriate  countermeasures. 

7.  Define  and  assess  the  anti-terrorism  environment  and  situation  to 
identify  and  quantify  the  threat. 

8.  Define  and  assess  the  anti-terrorism  environment  and  situation  by 
identifying  critical  assets  and  vulnerabilities  and  discuss  force  protection  employment  to 
protect  these  assets. 

9.  Define  the  assessment  zone  as  presented  in  the  Defense  in  Depth 
concept  and  in  accordance  with  Navy  AT/FP  TTP. 

10.  Demonstrate  the  application  of  the  assessment  zone  as  described  in  the 
Defense  in  Depth  concept  and  in  accordance  with  Navy  AT/FP  TTP. 

1 1 .  Define  the  warning  zone  as  described  in  the  Defense  in  Depth  concept 
and  in  accordance  with  Navy  AT/FP  TTP. 

12.  Demonstrate  the  application  of  the  warning  zone  as  described  in  the 
Defense  in  Depth  concept  and  in  accordance  with  Navy  AT/FP  TTP. 

13.  Define  the  threat  zone  as  described  in  the  Defense  in  Depth  concept 
and  in  accordance  with  Navy  AT/FP  TTP. 

14.  Demonstrate  the  application  of  the  threat  zone  as  described  in  the 
Defense  in  Depth  concept  and  in  accordance  with  Navy  AT/FP  TTP. 

15.  Define  the  purpose  of  pre-planned  responses  in  accordance  with  Navy 
AT/FP  TTP  and  NTTP  3-07.2.1  (Rev  A). 
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16.  Define  the  likely  terrorist  threats  by  applying  the  pre-planned 
responses  concept  in  accordance  with  Navy  AT/FP  TTP. 

2.  Performance  Standards 

Upon  completion  of  the  lesson  the  student  will  be  able  to: 

1 .  Successfully  define  the  importance  of  the  Defense  in  Depth  concept  in 
accordance  with  Navy  AT/FP  TTP. 

2.  Successfully  identify  the  definition  among  other  distracters  of  Defense 
in  Depth  concept  in  accordance  with  Navy  AT/FP  TTP. 

3.  Successfully  apply  the  procedures  of  the  Defense  in  Depth  concept  to 
ship  in-port  security  scenario  by: 

a.  Correctly  identify  port  vulnerabilities. 

b.  Correctly  identify  own  high  value  assets  that  need  protection. 

c.  Correctly  identify  own  critical  assets  vulnerabilities. 

d.  Mitigate  vulnerabilities  to  an  acceptable  risk. 

e.  Correctly  establish  a  warning  zone. 

f.  Correctly  establish  a  threat  zone. 

g.  Correctly  establish  an  assessment  zone. 

h.  Correctly  employ  force  protection  assets  to  protect  high  value 
assets. 

i.  Assessing  risk. 

j.  Develop  countermeasures. 

k.  Identify  and  quantify  the  threat. 

4.  Successfully  assess  the  anti-terrorism  environment  and  situation  given 
intelligence  reports  and  visual  aids. 
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5.  Correctly  define  the  purpose  of  pre-planned  responses  in  accordance 
with  Navy  AT/FP  TTP  and  NTTP  3-07.2. 1  (Rev  A). 

6.  Successfully  identify  the  definition  among  other  distracters  of  the 
purpose  of  pre-planned  responses  in  accordance  with  Navy  AT/FP  TTP  and  NTTP  3- 
07.2.1  (Rev  A). 

7.  Complete  an  examination  with  an  80%  correct  answer  rate  of  the 
following  terms: 

a.  Layered  defense  concept  definition  and  application. 

b.  Assessment  zone  definition. 

c.  Warning  zone  definition. 

d.  Threat  zone  definition. 

e.  Define  the  purpose  of  the  assessment,  warning  and  threat  zones. 

f.  Organization  of  the  Naval  Vessel  Protection  zone. 

g.  Assets  used  in  the  assessment,  warning  and  threat  zones. 

h.  Watch-stander  actions  in  the  assessment,  warning  and  threat  zones. 

i.  Watch-stander  determination  of  threat. 

j.  Definition  and  purpose  of  pre-planned  responses. 

k.  Identify  future  threats  as  defined  by  Navy  instructions. 

3.  Enabling  Objectives 

The  enabling  objectives  detennine  the  pre  requisite  skills  required  for  completion 
of  each  specific  learning  objective.  Enabling  objectives  define  the  execution  of  the 
required  tasks  that  accomplish  each  specific  learning  objective.  Enabling  objectives  for 
the  lesson  prepared  in  this  thesis  are  detailed  in  Appendix  B. 
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4. 


Media  Selection 


The  learning  objectives  can  be  presented  to  the  student  using  several  teaching 
strategies,  including  lecture  and  practical  exercises.  The  teaching  strategy  that  can  be 
used  can  be  either  classroom  lecture  or  CBT  lecture  with  simulation.  The  media  selection 
used  has  to  be  based  on  the  needs  of  the  Navy  and  the  technology  that  allows  the  proper 
implementation  of  the  learning  objectives.  A  detailed  explanation  of  the  methodology 
used  for  media  selection  can  be  found  in  Appendix  C. 

To  determine  which  media  can  be  used  for  a  Navy-wide  prototype,  the  following 
questions  need  to  be  assessed: 

1 .  How  many  people  need  the  training? 

2.  Is  classroom  instruction  viable  and  cost  effective? 

3.  Can  all  the  target  audience  participants  attend  the  training? 

4.  Do  the  participants  need  the  instruction  as  refresher  training? 

5.  Do  the  participants  need  immediate  feedback  from  subject  matter  experts? 

6.  Do  participants  have  adequate  technology  to  perform  the  training? 

7.  Will  the  training  be  self-  paced  or  sequenced  by  classroom  schedules? 

After  assessing  the  Navy’s  current  and  future  capabilities  it  has  been  determined 
that  the  lessons  can  be  viewed  using  any  current  Navy  Desktop  workstation  with  the 
following  minimum  requirements: 

Operating  system:  Windows,  Macintosh  or  Linux 

Video  Cards:  Standard 

Memory  512  MB  Single  Channel  DDR2  SDRAM 

Hard  Drive:  60  GB  Hard  Drive  (7200  RPM) 

Monitor:  Standard 

Optical  Drive:  24X  CD  ROM 

Sound:  Standard  speakers  or  standard  headphones 
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Mouse:  Standard,  ergonomic  design 


5.  Resources 

The  resources  used  for  the  development  of  the  lessons  were  provided  by  the 
Center  for  Security  Forces  (CENSECFOR)  1575  Gator  Boulevard,  Suite  226,  Norfolk, 
Virginia  23521-275 1 .  Additional  ressources  for  the  X3D  content  of  the  course  were 
provided  by  the  SAVAGE  research  group  in  the  MOVES  Institute  at  the  Naval 
Postgraduate  School. 

D.  DEVELOPMENT 

Lessons  were  constructed  using  the  instructional  material  currently  used  in  the 
ATO  Level  II  course  provided  by  CENSECFOR.  Due  to  the  experimental  subjects’  time 
constraints  and  availability  the  lessons  used  for  this  thesis  were  modified  from  those  used 
in  the  ATO  Level  II  course  to  accommodate  for  these  constraints  and  to  allow  for  the 
proper  completion  of  the  tested  lessons.  After  the  task  analysis  was  completed,  scenario 
storyboards  (detailed  storyboards  are  located  in  Appendix  D  and  Appendix  E)  were 
created  for  the  two  studies  that  are  covered  in  this  thesis.  During  media  selection  it  was 
determined  that  for  the  delivery  of  the  knowledge  objectives  of  the  course  is  used 
Articulate  Presenter  trial  version  software  and  use  the  SavageStudio  for  the  delivery  and 
performance  assessment  of  the  skill  objectives  in  the  study. 

Articulate  Presenter  trial  version  (www.articulate.com)  is  a  Flash  authoring  tool 
that  was  used  for  the  transformation  of  the  storyboards  into  Flash  media.  The  software 
allows  the  developer  to  add  visual  graphics,  notes  and  audio  to  the  storyboards  creating 
interactive  movies.  For  a  full  tour,  demos  and  an  overview  of  the  Articulate  software  are 
at  (http://www.articulate.com/products/presenter.php). 

SavageStudio  was  used  to  develop  the  scenarios  used  for  the  skill  objectives 
performance  and  assessment.  SavageStudio  was  used  in  the  scenario  generation  allowing 
the  user  to  create  complete  simulations  with  minimal  setup.  SavageStudio  provided  the 
scenario  creator  (student)  with  drag  and  drop  interfaces  for  minimum  preparation  on  how 
to  use  the  software. 
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The  created  prototype  used  for  data  collection  employs  a  mixed-mode  approach 
where  the  instructor  uses  both  CBT  and  X3D  graphics  for  instruction  and  scenario 
development.  The  subject  matter  expert  is  available  during  the  instruction  to  assist  and 
answers  any  student  questions. 

1.  Instructional  Materials 

The  storyboard  created  for  the  CBT  vs.  Cl  effectiveness  comparison  (Lesson  1)  is 
located  in  Appendix  D 

The  storyboard  created  for  the  CBT  effectiveness  (Lesson  2.1)  is  located  in 
Appendix  E. 

E.  SUMMARY 

This  chapter  provides  an  in-depth  look  at  the  analysis  (task  analysis  and  target 
audience),  design  (learning  objectives,  performance  standards,  enabling  objectives,  media 
selection,  and  resources)  and  development  of  the  developed  lessons  for  the  AT/FP  course 
using  the  ISD  framework.  These  lessons  can  be  used  for  the  AT/FP  course  in  conjunction 
with  the  SavageStudio  scenario  generation  tool.  The  ADDIE  model  was  used  to  ensure 
that  the  course  objectives  meet  the  ATO  Fevel  II  course  objectives.  This  methodology  is 
advisable  and  suitable  for  other  courses. 
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IV.  METHOD 


A.  INTRODUCTION 

This  chapter  provides  an  in-depth  look  at  the  two  studies  conducted  during  this 
thesis  research.  The  chapter  is  divided  in  the  following  manner:  study  I  AT/FP  CBT 
versus  Cl,  and  Study  II  Training  effectiveness  of  the  ATFP  CBT  with  Simulation.  The 
participants,  variables  measured,  apparatus  and  procedures  used  in  the  implementation 
are  presented. 


B.  STUDY  I 

1.  Participants 

The  pool  of  volunteer  participants  for  study  I  was  comprised  of  16  students  of  the 
OA  3402  course  and  other  MOVES  students  attending  the  Naval  Postgraduate  School. 
The  students  of  the  course  are  Navy  and  Marine  Corps  and  officers  ranks  range  from  0-2 
to  0-4  male  and  female. 


Study  1  Participant's  Information 

RANK 

RATE 

GENDER 

AGE 

SERVICE  Yrs,  Mo. 

0-3 

USN,  SWO 

F 

34 

10 

0-4 

USN,  PILOT 

M 

35 

13.9 

0-2 

USN,  HR 

M 

25 

3.6 

0-3 

USN,  SWO 

M 

34 

16.2 

0-3 

USN,  SWO 

M 

27 

4.9 

0-4 

USN,  PILOT 

M 

33 

11 

0-3 

USN,  SWO 

M 

32 

14 

0-4 

GERMAN,  ARMY 

M 

27 

6 

0-4 

USN,  IP 

M 

34 

15 

0-3 

USN,  IP 

M 

34 

14.6 

N/A 

CIV 

M 

35 

10 

N/A 

CIV 

M 

33 

10 

0-4 

GERMAN,  NAVY 

M 

32 

12 

0-3 

USN,  HR 

M 

31 

8 

0-3 

USN,  HR 

F 

29 

7 

Table  1.  Study  I  Participant’s  information. 
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The  participant’s  previous  assignments  include  Navy  ships,  Navy  submarines, 
shore  installations,  and  aviation  commands  ashore  and  afloat.  The  treatment  of  subjects 
was  in  accordance  with  the  ethical  standards  of  the  American  Psychological  Association 
(APA). 


2.  Experimental  Design 

The  independent  variable  for  this  experiment  is  the  mode  of  delivery,  media,  of 
instruction.  Two  conditions  of  the  independent  variable  are  considered,  CBT  and  CI.  The 
dependent  variable  was  post-test  scores.  Scores  were  taken  after  completion  of  the 
training.  Demographics  of  interest  include:  age,  gender,  military  branch,  type  and  number 
of  past  duty  assignments  Gobs),  education  level,  learning  style  and  computer  skills. 

The  purpose  of  study  I,  was  to  determine  if  there  is  a  difference  in  perfonnance 
between  groups  that  received  lessons  with  the  same  content  using  different  media 
delivery,  namely  CBT  and  CI.  Answering  this  question  is  a  first  step  in  detennining  if 
CBT  is  a  viable,  cost-effective  and  flexible  alternative  to  CI  training  for  AT/FP  training 
requirements.  Study  details  follow. 

A.  All  assessments  where  done  using  printed  material. 

B.  The  following  experimental  design  was  used  to  compare  CBT  vs.  CI  media 
effectiveness.  Participants  received  AT/FP  training  using  different  media  delivery 
methods  CI,  CBT.  Pre-  and  post-tests  were  conducted  to  examine  mean  differences 
between  groups  after  training  was  completed.  Table  2  shows  the  experimental  design  for 
Study  I. 
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Instructional  Pre-Test  Treatment  Post-Test 

Media 


CBT 

X0 

Oj 

X! 

Cl 

X0 

02 

X! 

Table  2.  Study  I  Experimental  Design.  Oi  refers  to  CBT  treatment  and  O2  refers  to  Cl 

treatment. 

Null  hypothesis:  Mean  score  of  post-test  for  the  CBT  group  equals  the  Mean 
score  for  the  post-test  of  the  Cl  group. 

Alternate  hypothesis:  Mean  score  of  post-test  for  the  CBT  group  does  not  equal 
the  Mean  score  for  the  post- test  of  the  Cl  group. 

3.  Apparatus 

The  classroom  training,  used  the  NPS  Human  Systems  Interaction  and  Learning 
(HSIL)  lab,  1  Dell  Precision  M90  computer  2.33GHZ  with  2.00G  RAM  that  contained 
the  slides  created  by  the  subject  matter  expert  (SME).  The  power  point  slides  were 
created  using  learning  and  enabling  objectives  used  in  the  Anti-terrorism  Officer  Course 
Level  II.  The  operating  system  for  this  computer  is  Windows  XP  Professional  Version 
2002  with  Microsoft  Power  Point  version  2003.  The  projector  is  Sony  VPL-X1000  with  a 
resolution  of  1024  X  768,  1100  lumens. 

The  CBT  portion  of  the  research  used  each  student’s  personal  laptop.  The  pre-test 
and  post-test  were  done  as  part  of  the  CBT.  The  CBT  will  be  created  by  the  SME  using 
the  ‘Articulate’  presenter  professional  software,  using  the  same  learning  and  enabling 
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objectives  used  in  the  creation  of  the  Power  Point  slides.  The  CBT  used  the  same  images 
and  text  as  the  Cl  power  point  and  included  a  voice  narration  of  the  infonnation  spoken 
by  the  instructor  in  the  CL 

4.  Procedure 

Participants  were  assigned  at  random  to  the  classroom  instruction  or  to  the  CBT 
instruction  groups  depending  on  the  last  digit  of  their  social  security  number.  Participants 
assigned  to  the  classroom  instruction  group  were  first  briefed  orally  and  in  writing  by  the 
subject  matter  expert  on  the  Institutional  Review  Board  (IRB)  process  and  then  asked  to 
complete  the  student  background  surveys.  After  completion  of  the  surveys,  the  students 
were  instructed  to  complete  a  pre-test  on  the  AT/FP  topic  of  instruction.  Upon 
completion  of  the  pre-test,  students  were  instructed  to  sit  in  a  classroom  where  classroom 
instruction  on  the  AT/FP  topic  was  conducted  using  an  SME  (this  author)  as  a  facilitator. 
After  completion  of  the  classroom  instruction,  students  were  instructed  to  conduct  a  post¬ 
test  on  the  AT/FP  topic  of  instruction. 

Participants  assigned  to  the  CBT  instruction  group  were  briefed  on  the 
Institutional  Review  (IRB)  process  and  asked  to  complete  the  student  surveys.  After 
completion  of  the  surveys,  the  students  were  instructed  to  complete  a  CBT  embedded 
pre-test  on  the  ATFP  topic  of  instruction.  Upon  completion  of  the  pre-test,  students  were 
instructed  to  proceed  with  the  CBT  lesson.  After  completion  of  the  CBT  lesson  students 
were  instructed  by  the  CBT  to  conduct  a  CBT  embedded  post-test  on  the  AT/FP  topic  of 
instruction. 

C.  STUDY  II 

1.  Participants 

The  pool  of  volunteer  participants  for  study  II  is  comprised  of  20  Officers  (2 
females  and  18  males)  of  the  following  Naval  and  Marine  Corps  communities:  Surface 
Warfare  Officers  (SWO),  Infonnation  Professionals  (IP),  Meteorological  Officers 
(METOC),  Communications  Officers  (COMMO,  USMC),  Pilots,  Naval  Flight  Officers 
(NFO),  Intelligence  Officers  (IW),  Human  Resources  (HR)  and  Medical  Officers  (MD), 
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all  attending  the  Naval  Postgraduate  School.  The  participant’s  previous  assignments 
include  Navy  ships,  Navy  submarines,  Navy  and  Marine  Corps  shore  installations  and 
aviation  commands  ashore  and  afloat.  The  participants  age  range  from  26  years  old  to  42 
years  old  with  an  average  age  of  33  years.  The  participants’  rank  from  LT/O-3  USN  or 
Capt./0-3  USMC  to  CDR  0-5  USN.  The  participants’  total  military  service  time  range 
from  5  years  to  19  years  of  military  service  time,  with  an  average  military  service  time  of 
12.63  years.  The  participants  reported  that  14  had  attended  AT/FP  training  courses  and  6 
had  not  attended  AT/FP  courses.  The  participants  reported  the  following  regarding  their 
perceived  knowledge  of  AT/FP:  3  participants  report  no  AT/FP  knowledge,  3  participants 
report  less  than  basic  AT/FP  knowledge,  1 1  participants  report  having  at  least  basic 
AT/FP  knowledge,  3  participants  report  having  reasonable  knowledge  (more  that  the 
basic  knowledge)  of  AT/FP,  and  no  participant  reported  having  expert  knowledge  of 
AT/FP.  Participants  also  reported  no  previous  AT/FP  training  in  the  last  12  months  while 
in  their  present  command.  Most  participants  reported  to  having  AT/FP  training  once  a 
year  in  their  previous  command,  however  some  reported  AT/FP  training  weekly,  monthly 
and  semi-annually  in  their  previous  commands.  The  treatment  of  subjects  was  in 
accordance  with  the  ethical  standards  of  the  APA. 
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Study  II  Participant's  information 

Rank 

Rate 

Gender 

Age 

Total  Time 
in  the 

Military 

Previous 

Knowledge 

0-4 

Commo 

male 

39 

12.3 

0 

0-3 

IP 

male 

29 

6 

1 

0-3 

Sub 

male 

36 

17.05 

1 

0-5 

Human 

Resources 

female 

42 

18.85 

1 

0-4 

NFO 

male 

34 

13.5 

1 

0-3 

Metoc 

male 

29 

8.15 

1 

0-3 

Metoc 

male 

37 

18.45 

1 

0-4 

SWO 

male 

37 

18 

0 

0-4 

Doc 

male 

34 

17 

1 

0-3 

Commo 

male 

34 

15.  8 

1 

0-3 

Metoc 

female 

32 

10 

1 

0-3 

Intelligence 

male 

35 

16 

1 

0-3 

SWO 

male 

27 

6 

1 

0-3 

IP 

male 

34 

14 

1 

0-3 

Pilot 

male 

31 

8.67 

0 

0-3 

Intelligence 

male 

35 

6 

0 

0-3 

IP 

male 

26 

5 

0 

0-3 

Commo 

male 

31 

14 

1 

0-3 

SWO 

male 

33 

16 

1 

0-3 

IP 

male 

37 

15 

0 

Table  3.  Study  II  Participant’s  information. 


2.  Experimental  Design 

The  purpose  of  study  II  is  to  determine  the  effectiveness  of  the  ATFP  instruction, 
namely  the  improved  performance  of  students  after  completing  the  ATFP  lesson  and 
simulation).  Answering  this  question  is  a  first  step  in  determining  whether  future  studies 
on  AT/FP  training  and  simulation  can  be  used  to  compare  against  classroom  instruction. 

The  independent  variable  for  this  experiment  is  the  treatment,  namely  combined 
AT/FP  lesson  and  simulation-based  training.  The  dependent  variable  for  this  experiment 
was  post-test  scores,  namely,  individual  knowledge  of  each  individual  on  the  AF/FP  topic 
(cognitive)  after  lesson  completion.  Another  variable  considered  is  the  number  of 
successfully  completed  tasks  for  a  particular  scenario  using  SavageStudio.  Scores  were 
taken  during  task  execution  and  evaluation  was  performed  after  completion  of  each 
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scenario.  Demographics  of  interest  include:  age,  gender,  military  branch,  type  and 
number  of  past  duty  assignments  (jobs),  education  level  and  computer  skills. 

A.  All  assessments  were  done  using  printed  material. 

B.  The  following  experimental  design  was  used  to  compare  the  effectiveness  of 
the  CBT  with  simulation.  Participants  received  AT/FP  training  using  CBT  with 
simulation.  Pre  and  post-tests  were  both  given  to  examine  learning  effects.  Table  4  shows 
the  study  II  experimental  design  parameters  of  interest 


Instructional  Observation  Observation 

Media 


CBT 

PRE-TEST 

POST-TEST 

SIMULATION 

Table  4.  Study  II  Experimental  Design. 


Null  hypothesis:  mean  score  of  pre-test  equals  the  mean  score  of  post-test. 

Alternate  hypothesis:  mean  score  of  pre-test  does  not  equal  the  mean  score  of 
post-test. 

3.  Apparatus 

For  the  learning  portion  of  the  study,  the  researchers  set  up  rooms  at  the  NPS 
library  with  1  Dell  Precision  M90  computer  2.33GHZ  with  2.00G  RAM  that  contained 
the  CBT  lessons  created  by  the  subject  matter  expert  (SME)  using  the  Articulate  trial 
software.  The  computer  provided  to  the  students  also  contained  the  SavageStudio 
software.  The  SavageStudio  software  can  be  downloaded  from  the  Scenario  Authoring 
and  Visualization  for  Advanced  Graphical  Environments  website  located  at 
https://savage.nps.edu/Savage/,  description  on  scenario  creation  using  SavageStudio  are 
provided  in  greater  detail  in  Chapter  V. 
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The  operating  system  for  this  computer  is  Windows  XP  Professional  Version  2002  with 
Microsoft  Power  Point  version  2003.  For  completion  of  the  IRB  forms  and  student 
questionnaires  the  participants  used  paper  forms. 

4.  Procedure 

The  students  (30  total)  received  e-mail  requests  to  participate  in  this  study.  Of  the 
30  students,  25  replied  back  with  interest  on  participating,  and  of  those  25  only  20  were 
able  to  complete  the  tasks  due  to  time  availability  and  conflicts  in  schedules.  Students 
were  instructed  to  report  to  the  NPS  library  at  designated  times  for  the  study.  Upon 
reporting  to  the  NPS  library,  each  student  was  briefed  on  the  Institutional  Review  process 
(IRB)  and  on  the  experimental  procedures,  written  and  referenced  in  Appendix  J  thru 
Appendix  N.  Students  were  asked  to  complete  the  student  survey.  After  completion  of  the 
surveys,  the  students  were  instructed  to  complete  a  pre-test  on  the  ATFP  topic  of 
instruction.  Upon  completion  of  the  pre-test,  students  were  instructed  to  complete  the 
CBT  instruction.  After  completion  of  the  CBT,  students  were  instructed  to  contact  the 
Subject  Matter  Expert  (SME)  for  further  instructions.  The  SME  directed  the  student  to 
read  the  scenario  information  sheet.  The  SME  directed  the  student  with  detailed  scenario 
generation  and  execution  instructions  and  then  asked  the  student  to  use  SavageStudio  to 
generate  and  execute  a  scenario.  The  following  paragraphs  provide  a  summary  of  the 
student  required  actions,  detailed  scenario  generation  and  execution  instructions  are 
provided  in  Chapter  V.  During  the  scenario  generation,  the  student  was  asked  to  select  a 
location  where  the  scenario  took  place.  Student  then  selected  the  assigned  friendly  assets 
HVU,  patrol  boxes,  patrol  boats,  radars  and  barriers.  Once  friendly  assets  were  selected, 
the  student  placed  the  assets  in  a  location  where  they  were  more  secure,  based  on  critical 
vulnerabilities  of  the  location  and  asset  capabilities.  The  student  was  asked  to  define  the 
assessment,  warning  and  threat  zones  by  using  the  assets  provided  namely  radars  and 
patrol  boxes.  Students  placed  and  modified  patrol  boxes  where  the  location  of  the  patrol 
box  would  increase  the  probability  of  the  intercept  of  hostile  assets  given  the  specific 
threat.  Once  the  patrol  boxes  were  completed,  the  student  assigned  patrol  craft  to  the 
assigned  zones.  This  step  was  repeated  for  a  second  patrol  box  and  a  second  patrol  craft. 

During  the  generation  of  the  scenario  the  SME  observed  the  student  to  see  if  he  was 
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completing  the  specified  tasks  as  specified  on  SavageStudio  Exercise  Student  Task 
Checklist.  The  SME  also  proceeded  to  ask  the  student  questions  of  how  the  placement  of 
his  assets  addresses,  port  vulnerabilities,  critical  asset  protection,  and  establishment  of  the 
assessment,  warning  and  threat  zones.  Once  the  student  generated  a  scenario  using 
SavageStudio,  the  SME  proceeded  to  ask  the  student  to  navigate  the  3D  scene  by  using 
the  viewpoint  button.  The  student  then  observed  the  execution  of  the  scenario.  The  SME 
provided  feedback  during  scenario  execution  regarding  the  student  placement  of  his 
assets  in  the  scenario.  The  student  launched  the  analyst  report  and  analyzed  scenario 
parameters  and  statistical  results  (boat  interceptions,  enemy  success  rate)  to  determine  if 
the  use  of  assets  was  optimized.  The  student  was  then  asked  to  develop  friendly  Courses 
of  Action  (COAs)  based  on  statistical  results  from  the  simulation  and  the  student  was 
allowed  to  go  back  to  the  scenario  generation  screen  to  implement  the  generated  COAs. 
Each  student  then  proceeded  to  observe  the  execution  of  the  scenario  and  analyzed  the 
results.  After  the  scenario  portion  of  the  study  was  completed,  the  student  was  asked  to 
complete  a  post-test. 

D.  SUMMARY 

This  chapter  provided  an  in-depth  look  at  the  two  studies  that  were  conducted 
during  this  thesis  research.  Detailed  information  is  provided  about  each  study,  namely 
Study  I  AT/FP  Computer-based  training  versus  Classroom  instruction,  and  Study  II 
Training  effectiveness  of  the  ATFP  CBT  with  Simulation.  Pertinent  infonnation  is 
presented  in  detail  for  each  study  including  participants,  variables  measured,  apparatus 
and  procedures  used  in  the  implementation  of  the  lessons. 
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V.  SCENARIO  CREATION 


A.  INTRODUCTION 

This  chapter  provides  an  in-depth  step-by-step  set  of  instructions  for  the  creation 
of  a  scenario  using  SavageStudio.  These  steps  include  launching  SavageStudio,  creating  a 
scenario,  adding  friendly  and  enemy  forces,  scenario  execution,  viewing  the  scenario,  and 
loading  a  previously  configured  scenario  from  disk. 

B.  LAUNCHING  SAVAGESTUDIO 

The  first  step  for  scenario  creation  is  to  launch  the  tool.  To  launch  the 
SavageStudio  application  the  user  clicks  on  the  SavageStudio  icon  and  launches  the 
application.  Upon  launching  the  application  a  command  window  with  scrolling  text 
appears  on  the  computer  screen  with  the  information  that  the  application  is  being 
initialized.  Once  SavageStudio  is  initialized  the  main  application  content  display  window 
is  displayed  in  the  computer  screen  as  shown  in  Figure  5. 


Figure  5.  SavageStudio  scenario  generation  display  window  presents  a  blank  palette  for 

scenario  configuration. 
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C.  SCENARIO  CREATION 

For  this  scenario  example  the  user  creates  a  scenario  located  in  Bremerton 
Washington,  then  adds  the  following  entities:  one  Nimitz  class  Aircraft  Carrier  that  acts 
as  our  High  Value  Unit  (HVU),  a  patrol  area  with  assigned  specific  dimensions,  two 
patrol  boats  for  perimeter  defense  assigned  to  patrol  inside  the  patrol  area,  and  two 
pleasure  boats  that  are  used  as  explosive-laden  vessels  by  the  hostile  terrorist  group. 

1.  Select  Location 

The  first  step  in  scenario  creation  is  to  select  a  port  from  the  available  listing  of 
locations  in  the  location  menu  selection  list.  Note  each  location  may  contain  more  than 
location  or  port.  Instructions  to  follow. 

•  First  place  the  arrow  pointer  over  the  locations  folder  to  select  a  location. 

•  Select  the  locations  folder  by  clicking  the  left  mouse  button  over  the  key 
icon  next  to  the  locations  folder  as  shown  on  Figure  6.  All  the  locations 
contained  in  the  folder  appear  on  the  screen  below  the  Locations  folder, 
as  shown  on  Figure  7. 

•  Select  a  specific  location  by  clicking  the  left  mouse  button  on  the  key 
icon  next  to  location,  select  Bremerton.  A  red  triangle  followed  by  the 
word  Bremerton  appears  on  the  computer  screen. 

•  By  clicking  the  left  mouse  button  over  the  key  icon  located  next  to  the 
word  Bremerton  the  map  for  Bremerton  appears  on  your  screen  as  shown 
on  Figure  8. 


42 


Figure  6.  SavageStudio  scenario  generation  display  window  with  the  location  menu 

selection  folder  opened. 
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Figure  7.  SavageStudio  scenario  generation  display,  with  left  window  depicting  location 

and  port  menu-selection  folders. 
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Figure  8.  SavageStudio  scenario  generation  display,  depicting  Bremerton  port  selection 
in  the  left  hand  pane  and  2D  display  of  Bremerton  in  the  right  hand  pane. 


2.  Add  Friendly  Forces 

The  next  step  for  scenario  creation  is  to  select  a  High  Value  Unit  and  to  add  other 
desired  friendly  forces  from  the  available  listing  of  robots,  ships,  tools,  communications, 
and  sensors  in  the  models  menu  selection  list.  Instructions  to  follow. 

•  First  place  the  arrow  pointer  over  the  ship’s  military  folder  to  select  a  high 
value  unit. 

•  Select  the  Ship’s  Military  folder  by  clicking  the  left  mouse  button  over  the 
key  icon  next  to  the  Ship’s  Military  folder  as  shown  on  Figure  9.  All  the 
ship  models  contained  in  the  folder  appear  on  the  screen  below  the  Ship’s 
Military  folder. 

•  Select  a  specific  HVU  by  clicking  the  left  mouse  button  on  the  key  icon 
next  to  Carrier  Nimitz  United  States.  A  white  paper  page  icon  followed  by 
the  word  Nimitz  CVN  appears  on  the  computer  screen. 
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•  By  clicking  the  left  mouse  button  over  the  key  icon  located  next  to  the 
word  Nimitz  CVN  the  ship  model  for  a  Nimitz  class  Aircraft  Carrier  is 
displayed  in  the  computer  screen  as  shown  on  Figure  10. 

•  The  model  for  the  Nimitz  CVN  is  attached  to  the  mouse  pointer  so  the 
model  moves  as  the  mouse  is  moved  over  the  application  screen. 

•  To  disengage  or  deselect  the  model  from  the  mouse  pointer,  press  the  Esc 
key  on  your  keyboard  by  disengaging  the  model  from  the  mouse  pointer 
model  movement  ceases  and  the  mouse  is  free  to  be  used  for  other 
purposes  on  the  application  screen. 


•  To  engage  or  select  the  model  again  place  the  arrow  pointer  over  the 
model  and  click  the  left  mouse  button,  by  completing  this  step  the  user  is 
again  allowed  to  move  the  model  and  place  it  anywhere  in  the  application 
screen  where  the  map  is  available. 
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Figure  9.  SavageStudio  scenario  generation  display  window  depicting  the  model, 
robots,  ships,  tools,  communications  and  sensors  menu  folders  and  sub-folders. 
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Figure  10.  SavageStudio  scenario  generation  display  window  depicting  a  Nimitz  class 
Aircraft  Carrier  selection  in  the  left  hand  pane  and  2D  display  of  a  Nimitz  class 
Aircraft  Carrier  in  the  right  hand  pane. 

•  To  manipulate  the  models  heading  meaning  to  rotate  the  model,  press  the 

up/down  arrow  keys  located  in  the  keyboard. 


•  Now  that  a  FIVU  is  selected  the  user  will  place  the  model  next  to  a  pier, 
note  that  the  heading  of  the  FIVU  is  not  the  same  as  the  pier  located  in  the 
scenario. 


•  Using  the  down  arrow  located  in  the  keyboard  change  the  model’s  heading 
to  match  the  pier’s  heading,  in  this  case  the  Nimitz  CVN  bow  points  south 
towards  the  water  as  shown  on  Figure  12. 
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Figure  1 1 .  SavageStudio  scenario  generation  display  window  depicting  a  Nimitz  class 

Aircraft  Carrier  rotating  next  to  a  pier. 


*  Stadia 


a  fl  C  v 


Figure  12.  SavageStudio  scenario  generation  display  window  depicting  a  Nimitz  class 


Aircraft  Carrier  located  next  to  a  pier  selection  in  the  left  hand  pane  and  the 
exposed  metadata  for  the  Nimitz  class  Aircraft  Carrier  in  the  right  hand  pane. 


Note  to  the  right  of  the  map  screen  the  model’s  information  (SMAL  metadata)  is 
displayed  while  the  model  is  selected.  This  information  can  be  modified  to  assign  a 
behavior  or  role  to  the  particular  model,  so  the  next  step  is  to  assign  the  HVU  behavior  to 
this  model.  The  default  value  for  a  model  behavior  parameter  is  set  to  patrol  craft. 

•  Place  the  arrow  pointer  in  the  model  information  screen;  scroll  down  until 
the  behavior  parameter  set  is  viewed  in  the  computer  screen. 
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•  Under  the  Behavior  parameter  set  there  is  a  select  box  with  the  word  type 
located  next  to  it,  select  the  folder  icon. 

•  Once  the  folder  icon  is  selected  a  dialog  box  is  displayed  on  the  computer 
screen,  it  has  a  set  of  behavior  libraries,  select  the  Savage  Defense  tactics 
folder  icon. 

•  Once  the  folder  icon  is  selected  a  dialog  box  is  displayed  on  the  computer 
screen,  this  folder  has  a  set  of  tactics,  select  the  friend  folder  icon. 

•  Once  the  folder  icon  is  selected  a  dialog  box  is  displayed  on  the  computer 
screen,  the  folder  has  a  set  of  individual  behavior  tactics,  select  the 
HVU.xml  icon. 

•  Once  the  behavior  is  selected  click  on  the  save  button,  the  dialog  box  will 
close  and  the  application  is  back  to  the  model  information  screen. 


Figure  13.  SavageStudio  scenario  generation  display  window  depicting  the  behavior 
libraries  selection  box  that  is  being  assigned  to  the  Nimitz  Aircraft  Carrier. 
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Figure  14.  SavageStudio  scenario  generation  display  window  depicting  the  behavior 
libraries  selection  box  and  the  friendly  assets  library  selection  sub-menu. 

The  next  step  is  to  select  patrol  zones.  For  this  example  one  patrol  zone  will  be 
created.  In  this  patrol  zone  the  user  places  the  military  assets  that  patrol  the  area  in  order 
to  protect  the  HVU.  Patrol  zones  are  used  to  determine  defensive  set  up  and  measures. 
FIVU  Patrol  zones  are  placed  in  the  location  where  our  assets  will  be  more  effective 
against  an  explosive  laden  vessel  threat. 

•  Place  the  arrow  pointer  over  the  patrol  zone  icon. 

•  Select  a  patrol  box  by  clicking  the  left  mouse  button  over  the  key  icon 
next  to  the  patrol  zone  folder  as  shown  on  Figure  15.  Select  a  patrol  box 
by  clicking  the  left  mouse  button  on  the  white  paper  page  icon  followed 
by  the  word  box,  a  red  patrol  box  appears  on  your  computer  screen. 

•  The  model  for  the  patrol  box  is  attached  to  your  mouse  pointer  so  the 
model  moves  as  you  move  the  mouse  pointer  over  the  application  screen. 

•  To  disengage  or  deselect  the  model  from  the  mouse  pointer,  press  the  Esc 
key  on  your  keyboard  by  disengaging  the  model  from  the  mouse  pointer 
model  movement  ceases  and  the  mouse  is  free  to  be  used  for  other 
purposes  on  the  application  screen. 
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•  To  engage  or  select  the  model  again  place  the  arrow  pointer  over  the 
model  and  click  the  left  mouse  button,  by  completing  this  step  the  user  is 
again  allowed  to  move  the  model  and  place  it  anywhere  in  the  application 
screen  where  the  map  is  available. 

Note  to  the  right  of  the  map  screen  the  patrol  zone  information  (SMAL  metadata) 
is  displayed  while  the  patrol  box  is  selected.  This  information  can  be  modified  to  assign 
the  patrol  zone  different  dimensions.  The  default  value  for  a  patrol  zone  is  set  to  50 
meters  in  the  X  direction,  0.01  meters  in  the  Y  direction  and  50  meters  in  the  Z  direction. 

•  For  this  example  the  user  makes  the  patrol  zone  250  meters  in  the  X 
direction  and  250  meters  on  the  Z  direction. 

•  Place  the  arrow  pointer  in  the  model  information  screen  select  the  X 
dimension  box  by  clicking  on  top  of  it.  Input  the  number  250  inside  the 
box  then  press  enter,  repeat  the  same  steps  for  the  Z  dimension,  after 
completion  of  this  steps  the  computer  screen  display  looks  similar  to 
Figure  15. 


Figure  15.  SavageStudio  scenario  generation  display  window  with  a  patrol  box  added. 
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The  next  step  is  to  select  patrol  assets  that  will  be  assigned  to  patrol  the  area 
inside  the  patrol  zone.  Available  assets  are  contained  in  the  model  folders.  For  this 
example  the  user  uses  two  patrol  boats.  The  following  steps  are  similar  to  the  steps  used 
to  place  the  HVU  and  the  patrol  zone  in  the  application  screen,  therefore  fewer  figures 
are  provided. 

•  Place  the  mouse  arrow  pointer  over  the  ship’s  military  folder  to  select  a 
patrol  asset. 

•  Select  the  ship’s  military  folder  by  clicking  the  left  mouse  button  over  the 
key  icon  next  to  the  ship’s  military  folder.  All  the  ship  models  contained 
in  the  folder  appear  on  the  computer  screen  below  the  ship’s  military 
folder. 

•  Select  a  specific  patrol  asset  by  clicking  the  left  mouse  button  on  the  key 
icon  next  to  Pearl  Harbor  security  boat.  A  white  paper  page  icon  followed 
by  the  word  Pearl  Harbor  security  boat  appears  on  your  computer  screen. 

•  By  clicking  the  left  mouse  button  over  the  key  icon  located  next  to  the 
word  Pearl  Harbor  security  boat  the  ship  model  for  a  security  boat  is 
displayed  in  your  computer  screen. 

•  The  model  for  the  Pearl  Harbor  security  boat  is  attached  to  your  mouse 
pointer  so  the  model  will  move  as  the  mouse  pointer  is  moved  over  the 
application  screen. 

•  To  disengage  or  deselect  the  model  from  the  mouse  pointer,  press  the  Esc 
key  on  your  keyboard  by  disengaging  the  model  from  the  mouse  pointer 
model  movement  ceases  and  the  mouse  is  free  to  be  used  for  other 
purposes  on  the  application  screen. 

•  To  engage  or  select  the  model  again  place  the  arrow  pointer  over  the 
model  and  click  the  left  mouse  button,  by  completing  this  step  the  user  is 
again  allowed  to  move  the  model  and  place  it  anywhere  in  the  application 
screen  where  the  map  is  available. 
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•  To  place  the  first  security  boat  in  the  application  screen,  place  the  model 
on  the  location  in  the  map. 

•  Click  the  left  mouse  button;  note  the  security  boat  is  still  attached  to  the 
mouse  pointer  to  place  the  second  security  boat  on  the  location  map  click 
the  left  mouse  button  as  shown  in  Figure  16. 

•  Press  the  Esc  key  to  deselect  the  model.  To  manipulate  the  models 
heading  as  previously  discussed  press  the  up/down  arrow  keys  located  in 
the  computer  keyboard. 


Figure  16.  SavageStudio  scenario-generation  display  window  depicting  a  Nimitz  class 
Aircraft  Carrier  and  the  two  added  patrol  boats  on  the  left  hand  pane  and  the 
selected  vehicle’s  metadata  on  the  right  hand  pane. 


Note  to  the  right  of  the  map  screen  the  selected  model’s  information  (SMAL 
metadata)  is  displayed.  As  previously  discussed  this  information  can  be  modified  to 
assign  a  behavior  or  role  to  the  particular  model.  The  default  value  for  a  model  behavior 
parameter  type  is  set  to  patrol  craft  so  for  the  two  patrol  boats  the  behavior  parameter 
type  does  not  need  to  be  changed. 
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•  The  patrol  boats  are  assigned  to  patrol  in  the  previously  created  patrol 
zone. 

•  Under  the  behavior  parameters  for  the  patrol  boat  to  the  right  of  the 
computer  screen,  there  is  a  white  icon  with  a  blue  cross  icon  labeled 
Zones. 

•  Click  once  on  the  icon  and  the  ProbabilityZoneGeometry  parameter  is 
made  available  for  modification  and  assignment,  four  parameters  are 
displayed  for  the  Zone.  Next  to  entitylD  a  number  is  displayed,  the 
number  displayed  is  used  to  identify  the  asset  in  the  simulation. 

•  Place  the  mouse  arrow  pointer  over  the  white  square  with  red  and  blue 
squares  connected  by  two  green  diamonds,  click  once  on  this  square,  the 
arrow  pointer  turns  into  the  entitylD  icon  that  was  recently  selected. 

•  Place  the  entitylD  icon  on  top  of  the  patrol  zone  left  click  the  mouse 
button  and  place  (drop)  the  entitylD  icon  in  the  patrol  zone  located  in  the 
location  map. 

3.  Add  Enemy  Forces 

The  next  step  for  scenario  creation  is  to  select  and  add  enemy  forces  from  the 
available  listing  of  robots,  ships,  tools,  communications  and  sensors  in  the  models  menu 
selection  list.  Instructions  to  follow. 

•  First  place  the  arrow  pointer  over  the  ship’s  civilian  folder  to  select  an 
enemy  asset. 

•  Select  the  ship’s  civilian  folder  by  clicking  the  left  mouse  button  over  the 
key  icon  next  to  the  ship’s  civilian  folder  as  shown  on  Figure  14.  All  the 
ship  models  contained  in  the  folder  appear  on  the  computer  screen  below 
the  ship’s  civilian  folder. 
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•  Select  a  specific  model  by  clicking  the  left  mouse  button  on  the  key  icon 
next  to  the  speed  boat.  A  white  paper  page  icon  followed  by  the  word 
speed  boat  with  driver  high-definition  appears  on  the  computer  screen. 


•  By  clicking  the  left  mouse  button  over  the  key  icon  located  next  to  the 
word  speed  boat  with  driver  high-definition  the  ship  model  for  a  speed 
boat  with  a  driver  is  displayed  in  the  computer  screen  as  shown  on  Figure 
17. 
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Figure  17.  SavageStudio  scenario  generation  display  window  depicting  a  Nimitz  class 
Aircraft  Carrier,  two  added  patrol  boats  and  two  speed  boats  on  the  left  hand  pane 
and  the  selected  vehicle’s  metadata  on  the  right  hand  pane. 


•  The  model  for  the  speed  boat  is  attached  to  your  mouse  so  the  model 
moves  as  the  mouse  pointer  is  moved  over  the  application  screen. 

•  To  disengage  or  deselect  the  model  from  the  mouse  pointer,  press  the  Esc 
key  on  your  keyboard  by  disengaging  the  model  from  the  mouse  pointer 
model  movement  ceases  and  the  mouse  is  free  to  be  used  for  other 
purposes  on  the  application  screen. 
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•  To  engage  or  select  the  model  again  place  the  arrow  pointer  over  the 
model  and  click  the  left  mouse  button,  by  completing  this  step  the  user  is 
again  allowed  to  move  the  model  and  place  it  anywhere  in  the  application 
screen  where  the  map  is  available. 

•  To  place  the  first  speed  boat  in  the  application  screen,  place  the  model  on 
the  location  in  the  map. 

•  Click  the  left  mouse  button;  note  the  speed  boat  is  still  attached  to  the 
mouse  pointer  to  place  the  second  speed  boat  on  the  location  map  click  the 
left  mouse  button  as  shown  in  Figure  17. 

•  Press  the  Esc  key  to  deselect  the  model. 

•  To  manipulate  the  models  heading  as  previously  discussed  press  the 
up/down  arrow  keys  located  in  the  computer  keyboard. 

Note  the  default  behavior  parameter  for  all  surface  ships  is  military  patrol  craft; 
therefore  the  behavior  needs  to  be  changed  for  the  two  enemy  speed  boats.  The 
explosive-laden  behavior  is  assigned  to  the  two  speed  boats.  To  the  right  of  the  map 
screen  the  model’s  infonnation  (SMAL  metadata)  is  displayed  while  the  model  is 
selected.  As  previously  discussed  in  the  HVU  model  example  this  information  can  be 
modified  to  assign  a  behavior  or  role  to  the  particular  model,  and  so  the  next  step  is  to 
assign  the  explosive-laden  vessel  behavior  to  this  model. 

•  Place  the  arrow  pointer  in  the  model  infonnation  screen;  scroll  down  until 
the  behavior  parameter  set  is  viewed  in  the  screen. 

•  Under  the  Behavior  parameter  set  there  is  a  select  box  with  the  word  type 
next  to  it,  select  the  folder  icon.  Once  the  folder  icon  is  selected  a  dialog 
box  is  displayed  on  the  computer  screen,  it  has  a  set  of  behavior  libraries, 
select  Savage  tactics  folder  icon. 
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Figure  18. 
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SavageStudio  scenario-generation  display  window  depicting  a  Nimitz  class 
Aircraft  Carrier,  two  added  patrol  boats  and  two  speed  boats  and  the  behavior 
libraries  selection  box  that  is  being  assigned  to  the  speed  boats. 


Once  the  folder  icon  is  selected  a  dialog  is  displayed  on  the  computer  screen,  it 
has  a  set  of  tactics,  select  the  hostile  folder  icon. 


Figure  19.  SavageStudio  scenario-generation  display  window  depicting  a  Nimitz  class 
Aircraft  Carrier,  two  added  patrol  boats,  two  speed  boats,  the  behavior  libraries 
selection  box,  and  Savage  Tactics  behavior  sub-folder  that  contains  the  behavior 
that  is  being  assigned  to  the  speed  boats. 
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Figure  20.  SavageStudio  scenario-generation  display  window  depicting  a  Nimitz  class 
Aircraft  Carrier,  two  added  patrol  boats,  two  speed  boats,  the  behavior  libraries 
selection  box,  Savage  Tactics  behavior  sub-folder  and  hostile  behavior  sub-folder 
that  contains  the  behavior  that  is  being  assigned  to  the  speed  boats. 

•  Once  the  folder  icon  is  selected  a  dialog  box  is  displayed  on  the  computer 
screen,  it  has  a  set  of  individual  behavior  tactics,  select  the 
ExplosiveLadenVessel.xml  icon. 

•  Once  the  behavior  is  selected,  click  on  the  save  button,  the  dialog  box  will 
close  and  the  application  will  proceed  to  the  model  information  screen. 

After  completion  of  the  previous  steps,  the  location  and  the  desired  entities  have 
been  placed  in  the  scenario.  The  entity  behaviors  have  been  modified  based  on  the  user 
preferences,  and  the  scenario  is  configured  to  be  run  either  visually  in  3D  or  off-screen 
for  statistics. 
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Figure  2 1 .  SavageStudio  scenario-generation  display  window  depicting  a  fully 

configured  scenario. 


4.  Scenario-Execution  Options 

The  user  has  two  scenario-executing  options  available  depending  on  what  the  user 
wish  to  gain  from  the  simulation.  The  first  execution  option  available  to  the  user  is  the 
3D  view,  which  allows  the  user  to  accomplish  specific  tasks  such  as:  visually  inspect  the 
entity  behaviors  and  verify  that  the  applied  tactics  are  according  to  pre-planned 
responses,  the  user  can  view  the  simulation  from  the  patrol  boat  point  of  view  to 
experience  what  the  patrol  boat  driver  is  receiving  by  visual  sensors  while  the  attack  is 
taking  place,  the  user  can  view  the  simulation  from  the  explosive-laden  vessel  point  of 
view  to  experience  what  the  speed  boat  driver  is  receiving  by  visual  sensors  while  the 
attack  is  being  conducted.  The  user  can  view  the  scenario  from  any  entity  point  of  view 
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or  from  a  top  view  (2D  geographic  view).  In  this  way  the  user  can  gain  visual  insight  to 
shortcomings  on  their  defensive  plans,  or  gain  insight  on  the  possible  shortcomings  on 
the  terrorist  attack  plan. 

The  second  execution  option  available  to  the  user  is  the  statistical  view.  This 
option  allows  the  user  that  is  no  longer  interested  in  visual  feedback  to  run  the  scenario 
faster  and  to  analyze  only  the  statistical  results.  The  user  can  iterate  over  thousands  of 
simulations  in  a  matter  of  seconds.  Some  of  the  tasks  that  can  be  accomplished  from  this 
statistical  view  are:  modifying  a  defensive  tactic  and  analyze  what  is  the  percentage  of 
success  (stopping  the  enemy  given  the  new  tactic  employment),  given  a  configured 
scenario  execute  the  scenario  one  thousand  times  and  see  the  percentage  that  the  enemy 
was  able  to  destroy  the  HVU  (enemy  success  percentage),  the  user  places  different  assets 
to  protect  the  HVU  and  analyzes  which  asset  is  better  given  the  current  scenario 
configurations. 

•  To  execute  the  scenario  in  a  3D  view  click  on  the  green  triangle  inside  the 
red  circle  icon  located  under  the  word  help. 

•  A  Save  As  dialog  box  appears  on  the  computer  screen,  name  and  save  the 
configured  scenario  as  shown  in  Figure  22. 


Figure  22.  SavageStudio  scenario-generation  display  window  depicting  a  fully 
configured  scenario  that  is  being  saved  to  the  scenario  libraries. 
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•  Once  the  scenario  is  saved,  the  application  launches  Viskit  assembly 
runner  and  the  simulation  viewer. 

•  On  the  Viskit  assembly  runner  screen,  click  on  the  gray  box  with  a  blue 
triangle  inside  it  to  start  the  simulation  as  shown  on  Figure  23. 

•  To  execute  the  scenario  for  a  statistical  view,  click  on  the  green  triangle 
inside  the  matrix  icon  located  next  to  the  3D  view  icon. 

•  A  save  as  dialog  box  appears  on  the  computer  screen,  name  and  save  the 
configured  scenario  as  shown  in  Figure  23. 

•  Once  the  scenario  is  saved  the  application  launches  Viskit  assembly 
runner. 


On  the  Viskit  assembly  runner  screen,  click  on  the  gray  box  with  a  blue 
triangle  inside  it  to  start  the  simulation  as  shown  on  Figure  24. 

SavageStudio  Viskit  assembly  runner  display  window  is  displayed  with 
the  simulation  results  on  the  right  hand  pane  of  the  screen  inside  the  Viskit 
assembly  runner  screen. 
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Figure  23.  SavageStudio  simulation  visual  display  window  depicting  a  fully  configured 
scenario  in  the  left-hand  window  and  the  Viskit  Assembly  Runner  in  separate 

right-hand  window. 
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Figure  24. 


SavageStudio  Viskit  Assembly  Runner  display  window  depicting  t 
execution  of  the  fully  configured  scenario. 
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5.  Scenario  Viewing  Options 

When  the  user  executes  the  scenario  in  3D  viewing  mode,  the  user  has  the  ability 
to  view  the  running  simulation  from  different  points  of  view.  The  user  can  view  the 
scenario  from  any  entity  point  of  view  or  from  a  top  view  (2D  geographic  view)  as 
previously  discussed.  The  user  can  navigate  and  explore  the  scene;  the  user  can  examine 
the  terrain,  buildings,  piers,  ships  and  all  the  features  available  in  the  simulated  3D  world. 
This  is  accomplished  by  using  different  viewpoints  accompanying  each  model  and 
created  by  the  application. 
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Figure  25.  SavageStudio  simulation  viewing  display  window  depicting  a  first-person 
view  of  the  simulation  from  the  explosive-laden  vessel. 


Figure  26.  SavageStudio  simulation  viewing  display  window  depicting  a  fully  configured 
scenario  in  the  right  hand  window  and  first  person  view  of  the  simulation  from  the 
explosive-laden  vessel  in  the  left  hand  window. 
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Figure  27.  SavageStudio  simulation  viewing  display  window  depicting  detonation  of  the 

explosive  laden  vessel  at  the  impact  point. 

D.  LOADING  A  PREVIOUSLY  CONFIGURED  SCENARIO 

Using  SavageStudio,  the  user  has  the  ability  to  both  configure  and  save  scenarios. 
These  saved  scenarios  can  be  loaded  at  a  later  time  for  viewing,  modification  and  analyst 
review.  If  the  user  wants  to  add  another  patrol  boat  to  the  previously  configured  scenario, 
the  user  does  not  need  to  start  a  new  scenario  from  the  beginning.  The  user  loads  a 
previously  configured  scenario  and  adds  the  new  entities  or  new  parameters  that  he  wants 
to  change  to  the  new  scenario;  this  feature  saves  the  user  time  when  minimal  changes  are 
required. 

•  To  load  a  scenario  from  disk,  the  user  launches  SavageStudio  once  the 
application  screen  is  loaded  to  the  desktop,  select  the  yellow  folder  icon. 

•  Once  the  folder  icon  is  selected  a  dialog  box  opens  on  the  computer 
screen,  select  the  desired  scenario  and  click  on  save. 

•  If  the  file  loads  satisfactorily  the  scenario  configuration  screen  is 
automatically  loaded  with  the  desired  scenario,  further  inspection  of 
parameters  and  addition  or  detection  of  entries  is  permitted. 

•  If  no  changes  are  required  then  execute  the  simulation  using  the  same 
steps  as  previously  discussed. 


63 


v  Savage  Studio 


File  View  Hel|i 


•M  035 

<►  □  Locations 
o-  C3  Bamers 
*•  Patrol  Zone 


t  n  Models 
+■  O  Robots 
►  C3  Ships  Civilian 
*-  Q  Ships  Military 
o-  n  Tools 

o-  □  Communications  And  Sensors 


Figure  28.  SavageStudio  scenario  generation  display  window  depicting  scenario  loading 

from  the  scenario  run  library. 
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Figure  29.  SavageStudio  scenario  generation  display  window  depicting  a  fully 

configured  scenario  loaded  from  the  run  library. 
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E.  SUMMARY 

This  chapter  provided  a  step  by  step  set  of  instructions  for  the  creation  of  a 
scenario  using  SavageStudio.  These  steps  included  launching  SavageStudio,  creating  a 
scenario,  adding  friendly  and  enemy  forces,  scenario  execution,  viewing  the  scenario  and 
loading  a  previously  configured  scenario  from  disk. 
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VI.  RESULTS  AND  DISCUSSION 


A.  INTRODUCTION 

This  chapter  provides  an  in-depth  look  at  the  two  studies.  Study  I  AT/FP 
Computer-based  training  versus  Classroom  instruction,  and  Study  II  Training 
effectiveness  of  the  ATFP  CBT  with  Simulation  results  and  discussion.  The  chapter  is 
divided  in  the  following  manner:  Study  I  results  and  discussion  then  Study  II  results  and 
discussion. 

Descriptive  statistics  are  used  to  compare  and  analyze  the  data  obtained  from  both 
studies.  Descriptive  statistics  describe  the  basic  features  of  the  data  and  this  evaluation 
method  enabled  us  to  present  the  data  in  an  unbiased  way.  This  approach  makes  it 
possible  to  determine  the  likelihood  that  the  differences  within  and  between  the 
participant  groups  were  caused  by  the  variables  studied  and  not  just  by  chance.  All  tests 
are  graded  on  a  scale  0-100. 

B.  STUDY  I  RESULTS 

The  analysis  of  the  Pre-Test  for  both  groups  indicated  that  the  Cl  students  had  a 
higher  mean  score  than  the  CBT  students.  The  CBT  mean  score  was  50.55  and  the  Cl 
mean  score  was  52.85.  The  analysis  of  the  Post-Test  for  both  groups  indicated  that  the 
CBT  students  had  a  higher  mean  score  than  the  Cl  students.  The  CBT  mean  score  was 
68.88  and  the  Cl  mean  score  was  60.71,  resulting  in  net  improvements  of  18.33  and  7.86, 
respectively. 
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Figure  30.  Pre-test  and  post-test  mean  scores  for  the  CBT  and  Cl  groups  in  study  I. 


The  difference  between  the  Pre-test  and  Post-test  scores  for  the  Cl  groups  reveal 
that  there  is  increase  in  performance  between  subjects  after  the  training  session  was 
conducted.  The  difference  between  the  Pre-test  and  Post-test  scores  for  the  CBT  groups 
reveal  that  there  is  increase  in  performance  between  subjects  after  the  training  session 
was  conducted. 

The  researcher  conducted  a  two  sample  T-Test  assuming  equal  variance  with  an 
alpha  =  0.05  and  14  degrees  of  freedom.  The  T  critical  value  found  is  Tc  =  2.144  and  the 
T0=  1.306  as  shown  on  Table  5. 

The  difference  between  the  Post-test  scores  for  both  the  CBT  and  Cl  groups 
reveal  improvement  in  performance  after  the  treatment  was  conducted.  However  due  to 
high  variance  in  the  data  further  statistical  analysis  revealed  that  there  is  no  significant 
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difference  in  perfonnance  in  regard  to  the  delivery  method  after  the  training  session. 
Therefore  T  critical  value  was  not  attained  then  we  have  to  accept  the  Null  hypothesis 
that  Mi  =  M2. 


Figure  3 1 .  CBT  participant’s  exam  scores  for  study  I. 


Classroom  Instruction  Test  Scores 


Figure  32.  Cl  participant’s  exam  scores  for  study  I. 
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Histogram 


□  Frequency 


Figure  33.  Frequency  distribution  of  student’s  pre/post-test  scores  difference  for  CBT 

participants. 


Histogram 


□  Frequency 


Figure  34. 


Frequency  distribution  of  student’s  pre/post-test  scores  difference  for  Cl 

participants. 
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CBT 

Classroom 

Mean 

18.33 

7.85 

Variance 

393.75 

65.47 

Observations 

9 

7 

Pooled  Variance 

253.06 

Hypothesized  Mean  Difference 

0 

Df 

14 

t  Stat 

1.30 

P(T<=t)  one-tail 

0.10 

t  Critical  one-tail 

1.76 

P(T<=t)  two-tail 

0.21 

t  Critical  two-tail 

2.14 

Table  5.  Two  Sample  T-Test  results  equal  variance  table  for  study  I. 


C.  STUDY  I  DISCUSSION 

The  results  show  statistically  significant  improvement  of  the  participant’s 
performance  on  the  tests  after  training  for  both  CBT  and  CL  Despite  the  apparent  higher 
scores  improvement  of  students  receiving  the  CBT  instruction  there  was  no  statistically 
significant  difference  between  the  increases  in  performance  based  on  the  delivery 
method.  Differences  in  performance  improvement  based  on  the  delivery  method,  can  be 
argued  that  for  this  research  none  is  found  due  to  the  small  sample  of  participants  or 
variability  of  the  participant’s  in  the  group.  Nevertheless  the  results  from  the  research 
given  the  circumstances  mentioned  and  the  literature  research  support  the  hypothesis  that 
when  the  material  presented  is  the  same,  CBT  is  as  good  as  CI.  The  practical  application 
is  that  results  from  this  study  support  the  continued  present  use  and  expanded  future  use 
of  CBT  by  the  U.S.  Navy. 

In  many  cases  CBT  is  the  best  media  because  it  allows  easier  access  to  the 
material.  If  it  is  on  the  Internet  or  on  CD’s  at  the  worksite,  then  training  can  be  done  on  a 
flexible,  short  notice  basis,  without  need  for  lengthy  scheduling.  In  addition  to  being 

more  flexible,  CBT  requires  less  time,  budget  and  facilities.  Time  savings  include; 
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eliminating  travel  time,  the  ability  to  start  and  stop  training  and  the  ability  to  skip 
material  already  mastered.  Budget  savings  include;  costs  to  maintain  classroom  facilities, 
instructor’s  salaries,  and  students  travel,  lodging  and  lost  work  costs.  Based  on  these 
advantages  and  increasing  constraints  on  time  and  money,  the  implication  is  that  similar 
training  for  the  U.S.  Navy  may  transition  to  CBT  in  the  future. 

D.  STUDY  II  RESULTS 

The  analysis  of  the  pre-test  and  post-test  score  data  for  the  group  that  received  the 
AT/FP  with  simulation  indicated  that  the  post-test  data  scores  are  higher  than  pre-test 
data  scores.  The  pre-test  mean  score  was  41%  and  the  post-test  mean  score  was  84%  a 
substantial  increase  in  performance.  The  difference  between  the  means  after  treatment  is 
43%  illustrated  in  Figure  35. 


Figure  35.  CBT  and  Simulation  pre/post-test  mean  scores  for  the  participants  for  study  II. 
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Score  Difference 


Figure  37.  Frequency  distribution  of  student’s  pre/post-test  scores  difference  for  study  II. 
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The  difference  between  the  pre-test  and  post-test  scores  for  the  group  reveal  that 
there  is  increase  in  performance  between  subjects  after  the  training  session  was 
conducted.  The  difference  between  the  pre-test  and  post-test  scores  can  be  attributed  to 
the  lesson  provided  for  the  students  in  the  group. 

A  T-Test  paired  two  sample  for  means  test  assuming  equal  variance  was 
conducted  with  an  alpha  =  0.05  and  19  degrees  of  freedom.  The  T  critical  value  is  found 
to  be  Tc  =  2.09  and  the  To=  2.07E-10  (effectively  zero)  as  shown  in  Table  6. 

The  difference  between  the  pre-test  and  post-test  scores  for  the  group  reveal  that 
there  is  a  significant  difference  in  performance  after  the  lesson  was  administered. 
Therefore  the  Null  hypothesis  that  Mi  =  M2  is  rejected  and  the  alternative  hypothesis  is 
accepted. 


Variable  1 

Variable  2 

Mean 

6.15 

12.6 

Variance 

7.29 

2.25 

Observations 

20 

20 

Pearson  Correlation 

0.483 

Hypothesized  Mean  Difference 

0 

df 

19 

tStat 

-12.15 

P(T<=t)  one-tail 

1.04E-10 

t  Critical  one-tail 

1.72 

P(T<=t)  two-tail 

2.07E-10 

t  Critical  two-tail 

2.09 

Table  6.  T-Test  paired  two  sample  for  student’s  mean  test  result  for  study  II. 


The  tasks  completion  for  the  scenario  portion  of  the  study  measuring  skill 
objectives  using  SavageStudio  reveals  that  20  of  20  participants  completed  the  objectives 
with  a  score  of  80%  or  better.  3  of  the  20  participants  achieved  a  score  of  83%  and  the 
other  17  participants  score  100%  completing  all  assigned  tasks.  The  scenario  task 
completion  mean  score  is  97.5%. 
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Tasks 

asseesment 

zone 

warning  zone 

threat  zone 

ID  critical 

Observe  the  scenario 

subject 

established 

established 

established 

assets 

Id  vulnerabilites 

and  explain  results 

score 

%  per  Student 

1 

1 

1 

1 

1  1 

1 

6 

1 

2 

1 

1 

1 

1  1 

1 

6 

1 

3 

1 

1 

1 

1  1 
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20 

20 

20 

20 

19 

18 

%  per  Objective 

1 

1 

1 

1  0.95 

0.9 

0.975 

Table  7.  Student’s  objective  completion  results  using  SavageStudio  in  Study  II. 


E.  STUDY  II  DISCUSSION 

The  results  show  statistically  significant  improvement  of  the  participant’s 
performance  on  the  specified  post-tests  after  training  was  conducted.  The  performance 
improvement  can  be  attributed  to  the  lesson  and  the  scenarios  completed  between  tests. 
Results  from  the  study  and  previous  research  on  the  subject  support  the  findings  that 
scores  on  post-test  will  be  different  than  scores  on  the  pre-test,  if  a  treatment  is  provided 
to  the  group  in  this  case  the  AT/FP  lesson  and  scenario.  The  practical  application  for  this 
study  is  that  this  research  can  be  used  to  support  future  research  where  CBT  and 
simulation  effectiveness  can  be  compared  with  Cl  effectiveness. 

If  future  research  finds  that  CBT  and  simulation  is  as  effective  as  classroom 
instruction  then  future  research  ought  to  be  conducted  on  the  effectiveness  and 
deployment  of  such  systems  for  future  Navy  training. 

F.  SUMMARY 

This  chapter  provides  an  in-depth  look  at  the  two  studies,  study  I  (AT/FP 
Computer-based  training  versus  Classroom  instruction)  and  Study  II  (Training 
effectiveness  of  the  ATFP  CBT  with  Simulation)  results  and  discussion.  Results  for  study 
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I  showed  that  there  is  no  significant  difference  between  CBT  and  Cl  if  the  content  is  the 
same.  Results  for  study  II  showed  that  there  is  a  difference  in  post-test  scores  after 
treatment  is  applied  to  a  group  (AT/FP  lesson  and  simulation). 
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VII.  CONCLUSION  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

This  thesis  was  motivated  by  the  needs  of  the  Navy  in  a  period  where  Anti- 
Terrorism  and  Force  Protection  Training  is  a  must,  time  and  resources  are  scarce,  and  the 
Navy’s  number-one  priority  is  to  train  its  sailors  in  an  efficient  manner.  The  enemies  of 
the  U.S.  recognize  that  terrorism  will  continue  to  create  problems  for  U.S.  Forces 
worldwide,  and  the  use  of  terrorist  tactics  will  continue  to  be  used  in  the  future  by  these 
groups. 

The  result  of  this  research  can  help  future  development  of  CBT  and  simulation 
training  systems  by  demonstrating  a  formal  methodology  to  accomplish  organizational 
requirements.  The  research  conducted  provided  an  in-depth  look  at  the  analysis,  design, 
development,  implementation  and  evaluation  of  the  developed  lessons  for  the  AT/FP 
course  using  the  ISD  framework.  The  use  of  the  ISD  in  the  creation  process  of  a  CBT  is 
imperative  if  successful,  meaningful  instruction  is  required.  The  CBT  developed  for  this 
thesis  research  incorporated  multimedia  and  simulation  to  enhance  the  knowledge  gain 
and  knowledge  transfer.  Future  developers  should  remember  Clark’s  (1994)  theory  that 
well  developed  instruction  is  the  most  important  aspect  of  training,  more  than  the 
delivery  method  itself,  and  not  let  themselves  fall  in  the  trap  to  add  more  bells  and 
whistles  just  for  the  sake  of  “coolness”  if  the  content  of  the  instruction  is  not  well 
designed,  then  no  matter  what  delivery  method  is  used  the  instruction  will  not  achieve  its 
objectives. 

The  results  for  the  first  study  showed  that  there  is  no  statistically  significant 
difference  between  CBT  and  Cl  if  the  content  is  the  same.  Results  for  study  II  showed 
that  there  is  a  difference  in  post-test  scores  after  treatment  is  applied  to  a  group  (AT/FP 
lesson  and  simulation).  This  research  study  supports  the  theory  that  CBT  is  a  viable 
alternative  to  Cl,  and  also  supports  the  hypothesis  that  CBT  and  Simulation  improves 
user  learning  and  performance.  These  results  support  Clark’s  (1994)  theory,  stated  in  the 
introduction,  that  media  does  not  affect  learning  if  the  content  of  the  instruction  is  the 
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same.  Thus  both  this  research  and  the  literature,  namely  Greitier  (2002)  and  Sitzmann  et. 
al  (2006)  support  the  theory  that  performance  can  be  effectively  increased  using  either 
type  of  instruction  delivery  method. 

Results  and  research  also  support  the  Navy’s  vision  of  the  Training  Revolution 
(Executive  Review  of  Navy  Training  2001).  Leaders  face  budget  and  operational  time 
constraints  that  force  them  to  work  in  unconventional  ways  to  accomplish  the  Navy’s 
mission.  Computer-Based  Training  and  Simulation  is  one  of  the  unconventional 
approaches  that  offer  several  advantages  for  the  military  organization.  Such  type  of 
training  offers  leaders  savings  on  resources,  time  and  money.  The  necessary  supporting 
technological  infrastructure  is  already  in  place  on  Navy  ships  and  sore  bases.  The  goal  of 
CBT  is  not  to  eliminate  Classroom  Instruction  altogether  but,  rather  to  supplement  it  or 
make  available  the  instruction  to  the  sailor  when  other  methods  are  not  available. 

The  CBT  developed  for  this  thesis  research  incorporated  multimedia  and  X3D 
graphics  simulation  to  enhance  the  knowledge  gain  and  knowledge  transfer. 

B.  RECOMEND ATIONS  FOR  FUTURE  WORK 

1.  Human  in  the  Loop  for  Training 

During  the  execution  of  this  thesis  research,  agent  and  security  postures  during 
scenario  creation  and  execution  were  not  modifiable,  and  so  the  participant  was  only  able 
to  select  behaviors  or  security  postures  from  the  Savage  libraries.  It  is  recommended  that 
mission  decision  points  be  added  to  SavageStudio  scenarios  where  the  user  has  the  ability 
to  manipulate  agent  actions  or  modify  security  postures  at  critical  decision  points,  then  let 
the  simulation  execute  and  see  the  results  of  the  user  decisions.  This  feature  allows  the 
user  to  further  exercise  course  of  action  and  planning  skills.  During  the  execution  of  this 
thesis  research  the  SME  observed  the  participant’s  actions  during  scenario  creation  and 
execution  and  listened  to  the  participant’s  verbal  explanation  of  his  actions.  The  SME 
collected  the  participant’s  information  by  annotating  the  infonnation  in  a  notebook  for 
later  debrief  and  feedback  on  the  user  performance.  It  is  recommended  that  feedback 
screens  and  replay  options  be  provided  to  the  instructor  allowing  him  to  go  over  the  user 
decisions  and  use  the  features  to  provide  feedback  and  teaching  points  to  the  user.  During 
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the  execution  of  this  thesis  research  infonnation  was  provided  to  the  participant  using 
paper  created  documents  referenced  in  Appendix  J  thru  Appendix  M.  It  is  recommended 
that  infonnation  be  provided  to  the  user  by  creating  a  tab  section  during  scenario  creation 
that  contains  the  following  information:  Commanding  Officer’s  Guidance  tab,  ATO 
Mission  Objectives  tab,  Threat  Assessment  tab  and  Intelligence  Briefs  tab. 

These  tabs  can  be  updated  as  the  simulation  is  executed,  for  example,  new 
intelligence  information  is  made  available  to  the  user,  so  that  the  user  may  change  force 
protection  posture  based  on  the  new  intelligence. 

2.  Expand  the  Behavior  Libraries 

It  is  recommended  that  the  SavageStudio  be  enhanced  by  authoring  additional 
behavior  libraries  to  include  the  following  behaviors: 

1.  Ship’s  personnel  responses  against  low  slow  flyer. 

2.  Ship’s  security  force  personnel  responses  against  swimmer  attack. 

3.  Security  boat  responses  against  swimmer  attack. 

4.  Hostile  swimmer  attack  behavior. 

3.  Migrate  SavageStudio  Application  for  Shore  Facilities 

It  is  recommended  that  the  approach  and  framework  used  in  SavageStudio  be 
migrated  to  shore  facilities  for  the  protection  of  high  value  assets  in  land.  The 
technologies  used  in  the  creation  for  SavageStudio  are  well  suited  for  similar  AT/FP 
applications  for  shore  facilities.  Representation  of  shore  and  land  facilities  can  be  created 
with  technology  being  researched  by  the  Savage  Group  at  the  Naval  Postgraduate  School. 
The  behavior  libraries  can  be  easily  modified  and  adapted  to  behaviors  used  in  AT/FP  of 
land  facilities. 

4.  Design  of  Experiments  (DOE) 

In  order  to  test  specific  variable  effects  when  conducting  risk  and  cost  assessment 

there  exists  a  need  to  keep  all  variable  effects  constant  except  for  the  tested  variables. 

Currently  some  of  the  algorithms  used  in  SavageStudio  use  stochastic  processes  which 
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use  probabilistic  methods  to  generate  states  in  the  simulation.  While  this  is  good  for 
course  of  action  and  training  applications,  the  additional  ability  to  only  modify  one 
variable  at  a  time  is  good  for  testing  risk  assessment.  For  example,  if  a  friendly  defense 
posture  is  maintained  constant  and  the  departure  of  the  explosive  laden  vessel  constant, 
the  user  can  modify  the  location  of  security  assets  between  runs,  and  then  the  analyst  can 
determine  if  the  modification  of  location  of  these  assets  benefited  or  hampered  a  given 
defense  plan. 


5.  Mission  Playback 

In  order  to  provide  trainees  with  feedback  there  is  a  need  to  add  a  mission 
playback  feature,  so  that  either  instructor  or  trainees  can  go  back  and  review  specific 
decision  points  during  the  simulation  execution. 

6.  Larger  Sample  Size 

It  is  recommended  that  this  study  be  replicated  with  a  larger  sample  size  and  true 
random  assignment  to  treatment  groups.  Future  research  that  builds  on  Clark’s  theory 
testing  varying  levels  of  complexity  of  the  material  presented.  There  may  be  a  point 
where  CBT  and  Cl  do  not  perform  equally,  and  indeed  a  well-designed  CBT  might 
outperform  Cl,  as  indicated  in  Study  I.  Maximizing  CBT  effectiveness  deserves  serious 
efforts  since  the  potential  payoffs  are  so  significant. 
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APPENDIX  A.  TASK  ANALYSIS  FOR  SAMPLE  LESSON  2  OF 
THE  ATO  LEVEL  II  COURSE  USING  CBT  AND  X3D  GRAPHICS 


Task  Analysis: 

The  following  task  analysis  identifies  the  required  tasks  that  the  user  will  be  expected  to 
perform  after  completion  of  the  lesson. 

Training  Objectives: 

Course  training  objective  2.0: 

Develop  and  implement  Pre-Planned  Responses  that  will  be  effective  mechanisms  to 
detect,  deter,  defend  and  mitigate  likely  terrorist  threats  against  U.S.  Navy  assets  and 
installations. 

Course  training  objective  3.0: 

Assess  the  anti-terrorism  environment,  situation  for  an  afloat  command,  given  current 
contingencies  and  sources  of  information. 

The  learning  objectives  that  will  be  measured  will  be  knowledge  and  skill  based 
objectives. 

Learning  objectives: 

2.1  Define  the  importance  of  the  Defense  in  Depth  concept  in  accordance  with  Navy 
Tactics,  Techniques  and  Anti-terrorism/Force  Protection  procedures  and  apply 
this  concept  to  ship  in-port  security.  (Knowledge)  Perfonnance  data  for  this 
objective  will  be  collected  in  the  post-test. 

2.2a  Define  the  principles  of  the  Defense  in  Depth  concept  in  accordance  with  Navy 

Tactics,  Techniques  and  Procedures  Anti-terrorism/Force  Protection  and  apply 
this  concept  to  ship  in-port  security  scenario.  (Knowledge)  Performance  data  for 
this  objective  will  be  collected  in  the  post-test. 

2.2b  Use  the  SavageStudio  in  visual  mode  to  explain  the  principles  of  defense  in  depth. 
(Knowledge)  Perfonnance  data  for  this  objective  will  be  collected  using  the 
SavageStudio,  student  will  explain  the  defense  in  depth  concept  as  it  applies  to  the 
scenario. 

2.2c  Use  the  SavageStudio  in  visual  mode  to  assess  and  demonstrate  the  application  of 
the  principles  of  defense  in  depth  as  they  occur  in  a  scenario.  (Skill)  Performance 
data  for  this  objective  will  be  collected  using  the  SavageStudio,  student  will 
assess  the  scenario  situation  and  will  demonstrate  the  application  of  the  defense  in 
depth  principles  by  the  correct  placement  of  assets  as  it  applies  to  the  scenario. 
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2.3a  Define  the  Assessment,  Warning  and  Threat  Zones  as  used  in  the  anti-terrorism 

program.  (Knowledge)  Perfonnance  data  for  this  objective  will  be  collected  in  the 
post-test. 

2.3b  Use  the  SavageStudio  visual  mode  to  demonstrate  the  assessment,  warning  and 
threat  zones  as  used  in  the  anti-terrorism  program  concept  of  defense  in  depth. 
(Knowledge)  Perfonnance  data  for  this  objective  will  be  collected  using  the 
SavageStudio,  student  will  explain  or  address  the  3  zones  as  they  apply  to  the 
scenario. 

2.3c  Use  the  SavageStudio  in  visual  mode  to  demonstrate  the  application  of  the 

Assessment,  Warning  and  Threat  Zones  as  used  in  the  anti-terrorism  program. 
(Skill)  Performance  data  for  this  objective  will  be  collected  using  the 
SavageStudio,  student  will  demonstrate  or  identify  the  3  zones  as  they  apply  to 
the  scenario. 

2.4a  Define  the  purpose  of  pre-planned  responses  in  accordance  with  Navy  Tactics, 
Techniques  and  Anti-terrorism/Force  Protection  and  NTTP  3-07.2.1  (Rev  A). 
(Knowledge).  Performance  data  for  this  objective  will  be  collected  in  the  post¬ 
test. 

2.4b  Define  the  likely  terrorist  threats  by  applying  the  pre-planned  responses  concept 
in  accordance  with  Navy  Tactics,  Techniques  and  Procedures  Anti¬ 
terrorism/Force  Protection.  (Knowledge)  Performance  data  for  this  objective  will 
be  collected  in  the  post-test. 

2.5a  Use  the  SavageStudio  in  visual  mode  to  identify  critical  assets,  and  vulnerabilities 
for  the  NAVSTA  scenario.  (Knowledge)  Performance  data  for  this  objective  will 
be  collected  using  the  SavageStudio,  describe  and  explain  the  assessment  of  the 
environment  and  situation,  the  student  will  explain  potential  threats  and 
vulnerabilities  as  they  apply  to  the  scenario. 

2.5b,  Use  the  SavageStudio  in  visual  mode  to  explain  the  anti-terrorism  environment 
and  situation;  using  the  X3D  visual  scene  of  the  generated  scenario  explain  the 
force  protection  employment  of  assets  to  protect  given  critical  assets  while 
minimizing  vulnerabilities.  (Skill)  Performance  data  for  this  objective  will  be 
collected  using  the  SavageStudio,  the  student  will  identify  potential  threats  and 
vulnerabilities  as  they  apply  to  the  scenario. 
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Task  1: 

At  the  end  of  task  1  the  user  will  be  expected  to: 

1 .  Complete  the  pre-test 

Task  1.  Complete  the  Pre-Test: 

1 . 1  Complete  20  question  test. 

1 .2  Inform  the  instructor  of  pre-test  completion. 


Task  2: 

At  the  end  of  task  2  the  user  will  be  expected  to: 

1.  Be  able  to  answer  questions  and  state  the  purpose  of  Defense  in  Depth. 

2.  Be  able  to  answer  questions  and  describe  the  principles  of  Defense  in  Depth. 

3.  Be  able  to  answer  questions  and  describe  the  Assessment,  Warning  and  Threat  Zones. 

4.  Be  able  to  answer  questions  and  describe  the  purpose  of  pre-planned  responses. 

5.  Be  able  to  answer  questions  and  describe  the  likely  terrorist  threats  for  developing 
pre-planned  responses. 

Task  2.  Complete  the  CBT  portion  of  the  lesson: 

2.1  Load  the  CBT  lesson  in  the  computer. 

2.2  Launch  the  AT/FP  CBT  course  using  the  mouse. 

2.3  Select  lesson  2  from  the  CBT  menu. 

2.3  Start  the  lesson. 

2.4  Observe  and  listen  to  the  lesson. 

2.5  Once  the  lesson  is  completed,  contact  the  SME  for  further  instructions. 


Task  3: 

At  the  end  of  task  3  the  user  will  be  expected  to: 

1.  Use  SavageStudio  in  visual  mode  to  assess  the  anti- terrorism  environment  and 
situation,  using  visual  scene  of  port  maps  to  identify  critical  assets  and  vulnerabilities. 

2.  Use  SavageStudio  in  visual  mode  to  describe  the  principles  of  defense  in  depth  as  they 
occur  in  a  scenario. 

3.  Use  SavageStudio  in  visual  mode  to  describe  the  assessment,  warning  and  threat  zones. 

4.  Use  SavageStudio  in  visual  mode  to  exercise  pre-planned  responses. 

5.  Set  up  defense  in  depth  for  the  NAVSTA  scenario  and  list  and  observe  the  pre-planned 
responses  for  boat-bom  IED. 

Task  3.  Use  SavageStudio  to  generate  and  execute  a  scenario: 

3.1  Read  the  scenario  information  given  by  the  instructor. 

3.2  Identify  critical  assets  and  vulnerabilities. 

3.3  Prioritize  identified  critical  assets. 

3.4  Select  location  to  place  critical  assets  for  maximum  security. 

3.5  Set  up  defense  in  depth  using  patrol  boxes,  patrol  boats,  radars  and  barriers. 

3.5.1  Place  the  patrol  box  within  the  chosen  harbor. 

3.5. 1. 1  Assign  the  friendly  asset  to  the  patrol  zone. 

3.5.2  Modify  the  patrol  zone  dimensions. 
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3.5.3  Place  barriers  within  the  chosen  harbor. 

3.5.4  Place  radar  assets  within  the  chosen  harbor. 

3.5.5  Repeat  task  3.5.1  for  a  second  friendly  asset  (patrol  boat). 

3.6  Execute  the  scenario  using  the  visual  mode. 

3.7  Navigate  the  scene  using  the  viewpoint  button. 

3.7. 1  Observe  patrol  boat  pre-planned  response. 

3.7. 1 . 1  Observe  patrol  boat  intercept  unauthorized  vessels. 

3.7. 1.2  Observe  patrol  boat  shoulder  unauthorized  vessels. 

3.7. 1.3  Observe  patrol  boat  or  SSDF  engage  unauthorized  vessel 
with  destructive  fire. 

3.8  Launch  and  review  the  analyst  report. 

3.9  Analyze  threat  and  simulation  parameters  using  the  analyst  report. 

3.9.1  Determine  patrol  boat  interceptions. 

3.9.2  Detennine  enemy  success  percentage. 

3.9.3  Detennine  if  use  of  assets  was  optimized. 

3.10  Develop  friendly  courses  of  action  based  on  statistical  results  from  the 
simulation. 


Task  4: 

At  the  end  of  task  4  the  user  will  be  expected  to: 

1 .  Complete  the  post-test 

Task  4.  Complete  the  Post-Test: 

4.1  Complete  20  question  test. 

4.2  Infonn  the  instructor  of  post- test  completion. 
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APPENDIX  B.  LESSON-ENABLING  OBJECTIVES 


LESSON-ENABLING  OBJECTIVES 

1.  Enabling  Objectives  that  will  Accomplish  the  Second  Learning 

Objective 

1.  Use  the  SavageStudio  visual  mode  to  describe  the  principles  of  defense 
in  depth  as  they  occur  in  a  scenario. 

2.  Use  the  SavageStudio  visual  mode  to  describe  the  assessment,  warning 
and  threat  zones  as  used  in  the  anti-terrorism  program  concept  of  defense  in  depth  in 
accordance  with  Navy  Tactics. 

3.  Use  the  SavageStudio  visual  mode  to  describe  the  Naval  Vessel 
Protection  Zone  in  accordance  with  Navy  Tactics. 

4.  Use  the  SavageStudio  visual  mode  to  describe  the  purpose  of  pre¬ 
planned  responses  in  accordance  with  Navy  Tactics. 

5.  Use  SavageStudio  to  analyze  course  of  action  (COA)  development  and 
implementation  of  Pre-Planned  responses,  identification  of  critical  assets,  identification 
and  quantification  of  threat,  assessing  asset  vulnerability,  assessing  risk  and  developing 
c  ountermeasures . 

6.  Use  the  SavageStudio  visual  mode  to  compare  and  select  COA  in 
accordance  with  Navy  Tactics. 

2.  Enabling  Objectives  that  will  Accomplish  the  Third  Learning 

Objective: 

1 .  Use  the  SavageStudio  to  visual  mode  to  assess  the  anti-terrorism 
environment  and  situation,  using  visual  scene  of  port  maps  to  identify  critical  assets  and 
vulnerabilities. 

2.  Set  up  defense  in  depth  for  the  NAVSTA  scenario  and  list  and  observe 
the  pre-planned  responses  for  boat-born  IED. 
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3.  Enabling  Objectives  that  will  Accomplish  the  Fourth  Learning 
Objective: 

1 .  Use  the  SavageStudio  visual  mode  to  analyze  COAs  that  will  be  used  in 
the  development  of  the  AT/FP  plan. 
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APPENDIX  C.  MEDIA  SELECTION 


The  methodology  used  in  media  selection  is  described  below  for  knowledge- 
based  objectives  and  skill-based  objectives.  The  process  was  repeated  for  each  objective 
to  select  the  most  effective  media  delivery  method. 

A.  KNOWLEDGE-BASED  OBJECTIVE 

1.  Define  the  objective 

Define  the  importance  of  defense  in  depth 

2.  Classify  the  objective 

Facts  about  defense  (declarative  knowledge  objective) 

3.  Strategy 

List  and  explain  important  reasons 

4.  Media  Delivery 

Classroom  lecture,  CBT,  textbook,  audio  visual 

5.  Content 

List,  narrative  explanation,  case  examples 

6.  Performance  Measurement 

Score  on  written  test,  instructor  evaluation 

B.  SKILL-BASED  OBJECTIVE 

1.  Define  the  objective 

Assess  and  demonstrate  the  application  of  defense  in  depth  as  it  occurs  in  a 
scenario 

2.  Classify  the  objective 

Tactical  concepts  about  defense  in  depth  (skill  objective) 
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3.  Strategy 

List  and  explain  operating  procedures,  show  how  to  implement  DID  (process) 

Demonstrate  correct  threat  ID,  set  defenses  IAW  with  threat  assessment, 
operate  the  SIM  and  execute  plan,  analyze  results  then  correct  tactical 
planning  and  execute  changes 

4.  Media  Delivery 

Classroom  lecture,  Scenario  and  Simulation 

5.  Content 

Narrative  explanation,  procedure  case  examples 

6.  Performance  Measurement 

SME  evaluation 
System  statistical  tally 
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APPENDIX  D.  STORYBOARD  FOR  TRAINING  AND 
CLASSROOM  INSTRUCTION  COMPARISON  STUDY 
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Slide  1  ® 

Slide  1 

Duration:  00:00:25 

Advance  mode:  Auto 

CENTER  FOR  SECURITY  FORCES 

Introduction  to  Terrorism 

Lesson  Topic 

1.0 

1-2-1 

Terrorism  is  nothing  new,  its  deadly  acts  have  been  around  for  hundreds  of  years  and  acts  of 
terrorism  against  Americans  and  other  nations  will  continue  for  many  years  to  come. 
Unfortunately  for  the  innocent,  terrorist  of  today  are  becoming  more  innovative  in  the  carrying 
out  of  their  deadly  acts.  The  end  result  are  scenes  of  massive  carnage. 


Slide  2  o 

Slide  2 

Duration:  00:00:05 

Advance  mode:  Auto 

References 

•  DOD  0-2000. 12-H 

•  NWP  3-07.2. 

•9-11  Commission  Report 

•  USATRADOC  DCINST  No.  1, 

Version  3.0 

1-2-2 

References  for  this  lecture  were  taken  from  the  following  sources: 


Slide  3  ® 

Slide  3 

Duration:  00:00:10 

Advance  mode:  Auto 

iL  )  Enabling  Objectives 

•  DEFINE  terrorism 

•  IDENTIFY  the  categories  of  terrorist 
organizations 

•  DESCRIBE  types  of  terrorist  training 

1-2-3 

No  notes  for  this  slide 
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Slide  4  ® 

Slide  4 

Duration:  00:00:17 
Advance  mode:  Auto 


Enabling  Objectives 


IDENTIFY  the  general  characteristics 

of  a  terrorist  operation 

LIST  the  seven  phases  of  a  terrorist 

operation 

DESCRIBE  common  terrorist  attacks 
DESCRIBE  the  types  of  terrorist  threats 
DESCRIBE  the  future  trend  of  terrorism 
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No  notes  for  this  slide 


Slide  5  €> 

Slide  5 

Duration:  00:00:18 
Advance  mode:  Auto 


“The  calculated  use  of  unlawful  violence  or  threat  of  unlawful 
violence  to  inculcate  fear;  intended  to  coerce  or  intimidate 
governments  or  societies  in  the  pursuit  of  goals  that  are 
generally  political,  religious,  or  ideological.” 

1-2-5 


You  will  take  command  at  a  time  in  which  events  combine  to  make  terrorism  a  top  national 
security  concern.  Since  the  Beirut  bombing  in  1983,  the  U.S.  military  has  suffered  more 
casualties  to  terrorism  than  in  combat  operations.  At  a  time  when  its  military  strength  is 
unmatched  as  the  world’s  remaining  superpower,  the  United  States  is  committed  to  more 
unfamiliar  environments  than  at  any  time  in  its  history  due  to  widespread  political  instability 
around  the  world.  Rogue  state  and  sub-state  opponents  of  U.S.  policy  cannot  challenge  U.S. 
military  power  directly,  but  they  can  and  do  attack  vulnerabilities  of  U.S.  Forces  in  an  effort  to 
erode  military  effectiveness  and  morale. 


Definition  of  Terrorism 

Terrorism  has  always  been  difficult  to  define.  Even  within  the  U.S.  Government,  different 
departments  adhere  to  different  definitions. 

The  DoD  defines  terrorism  as  “The  calculated  use  of  violence  or  threat  of  violence  to  inculcate 
fear;  intended  to  coerce  or  to  intimidate  governments  or  societies  in  the  pursuit  of  goals  that  are 
generally  political,  religious  or  ideological”. 
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Slide  6  ® 

Slide  6 

Duration:  00:00:36 

Advance  mode:  Auto 

1 fjtS 

■nJL'/  Common  Elements  of  Terrorism 

•  Political 

•  Psychological 

•  Violent 

•  Dynamic 

•  Deliberate 

1-2-6 

Elements  of  Terrorism: 

The  definition  in  the  previous  slide  is  clear  and  comprehensive.  It  contains  the  three  elements  that 
should  be  included  in  any  definition  of  terrorism: 

1.  Political  -  Clausewitz’  statement  that  “war  is  a  continuation  of  policy  by  other  means”  is  taken 
as  a  truism  by  terrorists. 

2.  Psychological  -  The  intended  result  is  to  create  a  psychological 

Terror 

3.  Violent-  Violence,  coercion,  and  destruction  are  used  in  the  commission  of  the  act  to  produce 
the  desired  effect 

4.  Dynamic  -  Terrorist  groups  demand  change,  revolution  or  political  movement. 

5.  Deliberate  -  Terrorism  is  an  activity  planned  and  intended  to  achieve  particular  goals. 

Slide  7  « 

Slide  7 

Duration:  00:00:10 

Advance  mode:  Auto 

^Categories  of  Terrorist  Organizations 

•  Government  affiliation  categories 

•  Motivation  categories 

•  Ideological  categories 

1-2-7 

These  topics  will  be  discussed  in  the  next  few  slides: 
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Slide  8  ® 

Slide  8 

Duration:  00:00:30 

Advance  mode:  Auto 

It-/'  Government  Affiliation 

•  Non-state  supported 

•  State  supported 

•  State  directed 

1-2-8 

Non-state  supported.  These  are  terrorist  groups  that  operate  autonomously,  receiving  no 
significant  support  from  any  government. 

•  State-supported.  These  are  groups  that  generally  operate  independently  but  receive 
support  from  one  or  more  governments. 

•  State-directed.  These  groups  operate  as  an  agent  of  a  government  and  receive  substantial 
intelligence,  logistic,  and  operational 

support  from  the  sponsoring  government. 

Slide  9  ® 

Slide  9 

Duration:  00:01:35 

Advance  mode:  Auto 

-  TV-  Motivation  Categories 

•  Separatist 

•  Ethnocentric 

•  Nationalistic 

•  Revolutionary 

1-2-9 

Separatist.  Separatist  groups  are  those  with  the  goal  of  separation  from  existing  entities  through 
independence,  political  autonomy,  or  religious  freedom  or  domination.  The  ideologies  separatists 
subscribe  to  include  social  justice  or  equity,  anti-imperialism,  as  well  as  the  resistance  to 
conquest  or  occupation  by  a  foreign  power. 

•  Ethnocentric.  Groups  of  this  persuasion  see  race  as  the  defining  characteristic  of  a  society,  and 
therefore  a  basis  of  cohesion.  There  is  usually  the  attitude  that  a  particular  group  is  superior 
because  of  its  inherent  racial  characteristics.  Examples:  KKK,  Aryan  Nation,  Black  Panthers, 

•  Nationalistic.  The  loyalty  and  devotion  to  a  nation,  and  the  national  consciousness  derived  from 
placing  one  nation’s  culture  and  interests  above  those  of  other  nations  or  groups  is  the  motivating 
factor  behind  these  groups. 

This  can  find  expression  in  the  creation  of  a  new  nation,  or  in  splitting  away  part  of  an  existing 
state  to  join  with  another  that  shares  the  perceived  “national”  identity. 

•  Revolutionary:  These  groups  are  dedicated  to  the  overthrow  of  an  established  order  and 
replacing  it  with  a  new  political  or  social  structure.  Although  often  associated  with  communist 
political  ideologies,  this  is  not  always  the  case,  and  other  political  movements  can  advocate 
revolutionary  methods  to  achieve  their  goals. 
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SlidelO 

Choices 

Interaction  type:  Choices 

Passing  score:  60 

Instructions:  None 

Custom  Message:  None 

7  7 

Learning  Game  Placeholder 

Learning  Game:  Choices 

Title:  Choices 

1  P 

Questions 

1 .  The  following  elements  are  essential  in  the  definition  of  terrorism  in  accordance  to  the 
Department  of  Defense: 

violence  or  threat  of  violence,  fear,  coercion,  pursuit  of  goals 

1)  True  (Correct) 

2)  False 

2.  Which  is  a  category  of  terrorist  organization? 

1)  government  affiliated  (Correct) 

2)  state  directed 

3)  separatist 

4)  anarchist 

3.  Which  is  a  characteristic  of  a  government  affiliated  terrorist  organization? 

1)  state  support  (Correct) 

2)  violence  fear 

3)  anarchist 

4)  IED 

Slide  11  ® 

Slide  11 

Duration:  00:00:06 

Advance  mode:  Auto 

CjL  Ideological 

•  Political 

•  Religion 

•  Social 

1-2-11 

Ideological  categories  describe  the  political,  religious,  or  social  orientation  of  the  group. 
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Political 


Slide  12  $ 

Slide  12 


Duration:  00:00:52 
Advance  mode:  Auto 


Right  Wing 
Left  Wing 
Anarchist 


Right  wing:  These  groups  are  associated  with  the  reactionary  or  conservative  side  of  the  political 
spectrum,  and  often,  but  not  exclusively,  are  associated  with  fascism  or  neo-  Nazism. 

Left  wing:  These  groups  are  usually  associated  with  revolutionary  socialism  or  variants  of 
communism. 

Anarchist:  Anarchist  groups  are  anti-authority  or  anti-government,  and  strongly  support  individual 
liberty  and  voluntary  association  of  cooperative  groups. _ 

Slide  13  © 

Slide  13 


Religious 


Duration:  00:00:13 
Advance  mode:  Auto 


Religiously  inspired  terrorism  is  on  the  rise,  with  a  forty-three  percent  increase  of  total 
international  terror  groups  espousing  religious  motivation  between  1980  and  1995.  While  Islamic 
terrorists  and  organizations  have  been  the  most  active,  and  the  greatest  recent  threat  to  the 
United  States,  all  of  the  major  world  religions  have  extremists  that  have  taken  up  violence  to 
further  their  perceived  religious  goals.  Religiously  motivated  terrorists  see  their  objectives  as  holy 
writ,  and  therefore  infallible  and  non-negotiable.  Religious  motivations  can  also  be  tied  to  ethnic 
and  nationalist  identities,  such  as  Kashmiri  separatists  combining  their  desire  to  break  away  from 
India  with  the  religious  conflict  between  Islam  and  Hinduism.  The  conflict  in  Northern  Ireland  also 
provides  an  example  of  the  mingling  of  religious  identity  with  nationalist  motivations.  There  are 
frequently  instances  where  groups  with  the  same  general  goal,  such  as  Kashmiri  independence, 
will  engage  in  conflict  over  the  nature  of  that  goal  (religious  or  secular  government). Christian, 
Jewish,  Sikh,  Hindu  and  a  host  of  lesser  known  denominations  have  either  seen  activists  commit 
terrorism  in  their  name,  or  spawned  cults  professing  adherence  to  the  larger  religion  while 
following  unique  interpretations  of  that  particular  religion’s  dogma.  Cults  that  adopt  terrorism  are 
often  apocalyptic  in  their  worldview,  and  are  highly  dangerous  and  unpredictable.  It  is  interesting 
to  note  that  religiously  motivated  terrorists  are  among  the  most  energetic  developers  of  Weapons 
of  Mass  Destruction  (WMD)  for  terrorist  use.  Also,  religiously  inspired  cults  executed  the  first 
confirmed  uses  of  biological  and  chemical  nerve  agents  by  terrorists. _ 
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Pro-Choice 


Often  particular  social  policies  or  issues  will  be  so  contentious  that  they  will  incite  extremist 
behavior  and  terrorism.  Frequently  this  is  referred  to  as  “single  issue”  or  “special  interest” 
terrorism.  Some  issues  that  have  produced  terrorist  activities  in  the  United  States  and  other 
countries  are: 

•  Animal  rights 

•  Abortion 

•  Ecology/environment 

•  Minority  rights 


Slide15 


Choices 

Interaction  type:  Choices 
Passing  score:  80 
Instructions:  None 
Custom  Message:  None 


Learning  Game  Placeholder 

Learning  Game:  Choices 
Title:  Choices 


Guestions 

1.  Which  is  included  in  a  ideological  affiliated  terrorist  organization? 

1)  religious  (Correct) 

2)  rationalist 

3)  anarchist 

4)  political 

2.  Which  is  a  political  affiliated  terrorist  organization? 

1)  anarchist 

2)  ideological 

3)  right  wing  (Correct) 

4)  PETA 

3.  These  are  included  in  a  government  affiliated  terrorist  organization  except: 

1)  separatist 

2)  ethnocentric  groups 

3)  revolutionary 

4)  political  (Correct) _ 
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Slide  16^ 

Immediate  Goals  and  Common 
Characteristics  of  Terrorist  Groups 

Duration:  00:00:17 

Advance  mode:  Auto 

© 

Immediate  Goals  and 
Common  Characteristics 
of  Terrorist  Groups 

1-2-16 

Terrorist  groups,  like  any  other  organization  have  both  immediate  and  long-range  goals. 

Therefore,  each  act  of  terrorism  is  designed  to  accomplish  something  specific  as  part  of  an 
overall  strategy.  These  immediate  or  short-term  goals  usually  include  one  or  more  of  the 
following:  (NEXT  SLIDE) 

Slide  17  « 

Slide  17 

Duration:  00:01:52 

Advance  mode:  Auto 

-  ft  Immediate  Goals 

■  Recognition 

■  Eliminate  western  influence 

•  Over-reaction  of  government 

■  Harass,  weaken,  or  embarrass  government 

■  Destroy  facilities,  disrupt  communications 

•  Discourage  foreign  investors 

■  Influence  government  decisions 

•  Free  prisoners 

■  Satisfy  vengeance 

1-2.17 

Obtain  worldwide  or  local  recognition  for  the  groups  cause.  The  publicity  advertises  the 
movement  and  is  of  utmost  importance  to  the  group.  Eliminate  western  influence  in  a  region. 
Osama  bin  Laden’s  goal  is  to  get  the  United  States  and  Israel  out  of  the  Middle  East  and 
establish  a  radical  fundamental  Islamic  regime  with  out  borders  throughout  the  Middle  East. 

Cause  an  over-reaction  by  the  government:  This  is  one  of  the  basic  precepts  of  revolutionary 
strategies.  Over-reaction  to  terrorist  activities  may  result  in  oppressive  and  restrictive  population 
control  measures,  such  as  instituting  special  antiterrorist  laws  that  allow  searches  warrants, 
detention  of  citizens  without  charges,  curfews,  and  travel  restrictions.  However,  people  see  these 
measures  as  a  violation  of  their  rights,  and  they  also  lend  an  element  of  legitimacy  to  grievances 
of  the  movement,  creating  an  atmosphere  more  sympathetic  to  the  terrorist  cause.  Harass, 
weaken  or  embarrass  government  security  forces.  Such  tactics  as  calling  in  false  alarms, 
assassinating  security  personnel,  and  conducting  sophisticated  raids  and  ambushes  are  used  to 
reduce  the  effectiveness  of  security  forces  and  erode  public  confidence  in  them  Obtain  money 
and  equipment.  Terrorist  acts  may  be  used  to  steal  or  extort  money  and  equipment,  especially 
weapons  and  ammunition  essential  to  the  struggle.  It  was  paid  in  1974  by  the  Exxon  Oil 

Corporation  for  the  release  of  Victor  Samuelson,  a  company  executive  working  in  Argentina  who 
had  been  held  for  144  days  by  the  Argentine  People’s  Revolutionary  Army.  Destroy  facilities  and 
disrupt  lines  of  communication.  This  tactic  obviously  hinders  the  government’s  ability  to  control 
the  violence,  both  militarily  and  politically.  It  creates  a  chaotic  atmosphere  within  the  population  in 
which  entire  Discourage  foreign  Investments  or  assistance  programs.  If  this  goal  is  achieved,  the 
economy  of  the  target  country  is  affected,  causing  an  increase  in  unemployment  and  a  decrease 
in  government  revenues.  Influence  government  decisions,  legislation  or  elections.  There  is  little 
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doubt  the  bombing  of  the  Marine  barracks  in  Beirut  influenced  our  government’s  decision  to 
eventually  withdraw  the  Marines _ _ 

Slide  18® 
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Duration:  00:00:38 
Advance  mode:  Auto 


Characteristics  of  Terrorist  Groups 
Terrorists  seek  to  intimidate  populations  by  promoting  and  spreading  fear.  They  are  militarily 
weaker.  This  means  they  must  resort  to  hit-and-run  guerrilla  tactics  and  will  be  reluctant  to  take 
up  a  static  defensive  posture  against  the  stronger  enemy  unless  they  possess  a  “bargaining  chip” 
such  as  hostages.  They  are  usually  urban  based.  This  offers  access  to  modern  means  of 
transportation,  money-laundering  mechanisms,  communications  systems,  and  international 
contacts.  Even  rural-based  insurgent  movements  will  have  urban-based  terrorist  cells  that  provide 
special  support  to  the  movement.  They  are  highly  mobile.  Being  urban-based,  the  terrorists  have 
access  to  the  transportation  systems  and  the  international  contacts  that  allow  them  to  obtain  the 
documents  they  need  to  travel  frequently  to  other  cities  and  countries  for  operations  or  in  search 
of  safe  havens  and  support.  They  usually  act  covertly.  While  the  clandestine  nature  of  the  terrorist 
groups  contributes  to  their  flexibility  and  effectiveness,  it  can  also  be  a  problem.  These  groups 
are  forced  to  maintain  and  support  intricate  operational  systems  complete  with  personnel  and 
assets. _ 

Slide  19® 

Typical  Training  Programs 

Duration:  00:00:37 
Advance  mode:  Auto 


Typical  training  programs  used  by  the  terrorist  groups  at  the  organizational  and  individual  levels 
Terrorist  training  programs  being  paramilitary  in  nature  are  oriented  to  the  mission,  the  physical 
characteristics  of  the  area  of  operations,  and  the  security  environment  in  the  operational  area. 
Most  of  the  training  is  somewhat  standardized.  Historically,  works  such  as  the  Mini-Manual  of  the 
Urban  Guerrilla  by  Brazilian  terrorist  Carlos  Marighella,  and  the  writings  of  revolutionary  leaders 
Mao  Tse-tung,  Che  Guevera,  and  Fidel  Castro  are  used  as  instructional  texts  and  indoctrination 
material.  Discipline  is  strictly  enforced,  and  trainees  are  closely  monitored  for  signs  of  weakness 
in  their  ability  or  ideological  commitment.  Internal  training  is  conducted  at  the  cell  level  by 
members  of  the  group  within  the  target  country,  usually  in  a  clandestine  fashion. 

External  Training:  Non-group  members  outside  the  operational  area  conduct  external  training.  It 


•  Internal 

•  External 

1-2-19 


Characteristics 


Seek  to  intimidate 
Militarily  weaker 
Urban  based 
Highly  mobile 
Well  trained 
Covert 
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can  take  two  different  forms.  First,  the  operatives  may  travel  abroad,  individually  or  as  a  group,  to 
training  camps  or  to  other  permanent  institutions  for  their  training. _ 

Slide  20® 

Typical  Training 

Duration:  00:00:21 
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A  typical  individual  terrorist  training  program  includes  most,  if  not  all  of  the  following: 

1.  Physical  conditioning. 

2.  Weapons  and  explosive  training,  including  how  to  purchase  or  manufacture  explosives  to 
construct  improvised  explosive  devices. 

3.  Political  indoctrination. 

4.  General  tactics  and  combat  techniques,  including  instruction  on  how  authorities  usually 
respond  to  hostage-barricade  situations. 5.  Trade/crafts,  including  placing  or  recovering  caches, 
opening  mail  covertly,  surreptitious  entry  to  homes  or  offices  and  wire-tapping. 

6.  Psychological  warfare. 

7.  Survival  skills,  including  survival  in  urban  and  foreign  environments. 8.  Communications, 
including  radio,  telephone,  “dead  drops”,  computers  and  satellite.  F.  Since  terrorist  operations  are 
paramilitary  in  nature,  it  is  not  surprising  that  there  are  many  similarities  in  the  training  of  terrorist 
and  in  the  training  of  Special  Operations  Forces  of  the  U.S.  In  fact,  terrorists  are  known  to  have 
capitalized  on  the  open  nature  of  our  society  and  our  willingness  to  publish  potentially  damaging 
material  such  as  the  Anarchist’s  Cook  Book,  Poor  Man’s  James  Bond  and  versions  of  the  Special 
Forces  and  Ranger  operational  manuals. 
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Learning  Game  Placeholder 

Learning  Game:  Choices 

Title:  Choices 

( 

1.  Which  2  training  programs  are  not  in  a  typical  terrorist  training  program? 

1)  Physical  condition,  survival  skills 

2)  Communications,  political  indoctrination 

3)  Electronic,  network  warfare  (Correct) 

4)  Weapons,  explosive  training 

2.  In  accordance  with  the  DoD  the  definition  of  external  terrorist  is: 

Non-group  members  outside  the  operational  area  conduct  external  training.  It  can  take  two 


Typical  Training 

Physical  conditioning 
Weapons  and  explosive  training 
Political  indoctrination 
Combat  tactics  and  techniques 
Psychological  warfare 
Survival  skills 
C  ommunications 
Specialized  training 
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different  forms.  First,  the  operatives  may  travel  abroad,  individually  or  as  a  group,  to  tr 

1)  True  (Correct) 

2)  False 
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Terrorist 
Operations 
and  it’s 

Planning  Cycle 


Will  be  discussed  in  the  next  few  slides 


Slide  23  « 

Slide  23 

Duration:  00:00:07 
Advance  mode:  Auto 


“The  main  point  is  to  select 
targets  where  success  is  100% 
assured.” 

Dr.  George  Habash,  Founder,  PFLP 
(Popular  Front  for  the  Liberation  of  Palestine) 


♦ 


Terrorist  Planning 
Cycle 


Broad 
T argot 
Sotection 


No  notes  for  this  slide 
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Duration:  00:00:35 
Advance  mode:  Auto 


Broad 

i  Target 

u 

Selection 

Collection  of  information  on  a  large  number 
of  potential  targets 

Also  includes  open  source  and  general  info 
collection 

-  Media  stories 

-  Internet 


•  Potential  targets  may  be  screened  based  on 

-  Symbolic  value 

-  Potential  for  high  media  value 

-  Objectives  of  group 


This  phase  is  the  collection  of  information  on  a  large  number  of  potential  targets,  some  of  which 
may  never  be  attacked,  or  seriously  considered  for  attack. 

This  phase  also  includes  open  source  and  general  information  collection.  Some  features  of  this 
type 

•Stories  from  newspapers,  other  media,  and  journalistic  sources  often  provide  key 
•Internet  research  provides  texts,  pictures,  blue  prints,  and  video  information. 

•Potential  targets  are  screened  based  on  symbolic  value  and  their  potential  to  generate  high 
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profile  media  attention. 

Slide  25  $ 

Slide  25 

Duration:  00:00:30 

Advance  mode:  Auto 

CD 

Intelligence 
ii  and 

Surveillance 

•  Targets  showing  potential  receive  a 
higher  priority 

-  Determine  if  target  is  beyond  group’s 
capability. 

-  Will  it  meet  intended  objectives? 

•  Information  gathered  include 

-  Practices/Procedures/Routines 

-  Residence  &  Workplace 

-  Transportation/Routes  of  Travel 

-  Security  Measures 

1-2-25 

Targets  showing  potential  vulnerabilities  are  given  a  higher  priority  of  effort. 

This  priority  establishes  the  requirement  to  gather  additional  information  on  the  targets’  patterns 
over  time. 

Slide  26  $ 

Slide  26 

Duration:  00:00:37 

Advance  mode:  Auto 

CD 

Specific 
in  Target 

Selection 

•  Selection  of  a  target  for  actual  operational  planning 
considers  some  of  the  following  factors: 

-  Does  success  affect  a  larger  audience  than  the 
immediate  victim(s)? 

-  Will  the  target  attract  high  profile  media  attention? 

-  Does  success  make  the  desired  statement  to  the 
correct  target  audience(s)? 

-  Is  the  effect  consistent  with  objectives  of  the  group? 

-  Does  the  target  provide  an  advantage  to  the  group 
by  demonstrating  its  capabilities? 

-  What  are  the  costs  versus  benefits  of  conducting 
the  operation? 

1-2-26 

No  notes  for  this  slide 
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Duration:  00:00:30 

Advance  mode:  Auto 

CD 

Pre-attack 
in  Surveillance 

and  Planning 

•  Detailed  and  often  long  term  surveillance  is 
conducted 

-  Conduct  security  studies. 

-  Conduct  detailed  preparatory  operations. 

-  Recruit  specialized  operatives  (if  needed). 

-  Procure  a  base  of  operations  in  the  target 
area  (safe  houses,  caches,  etc.). 

-  Design  and  test  escape  routes. 

-  Decide  on  type  of  weapon  or  attack. 

1-2-27 

No  notes  for  this  slide 
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Slide  28  ® 

Slide  28 


Duration:  00:00:14 
Advance  mode:  Auto 


Attack  plan  is 
developed 

-  Improve  the 
odds  of  success 

-  Confirm 
planning 
assumptions 

-  Develop 
contingencies. 


1-2-28 


No  notes  for  this  slide 
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Slide  29 


♦0: 


Duration:  00:00:34 
Advance  mode:  Auto 


•  Terrorists  conducting  planned  operations 
possess  important  tactical  advantages. 


-  Surprise 

-  Choice  of  time,  place,  and  conditions  of  attack. 


Employment  of  diversions  and  secondary  or 
follow-up  attacks. 


Employment  of  security  and  support  positions 
to  neutralize  target  reaction  forces  and  security 
measures. 


-  It  may  be  months  before  planned-for  circumstances 
occur  and  the  attack  takes  place 

1-2.29 


No  notes  for  this  slide 
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Slide  30 


Duration:  00:00:33 
Advance  mode:  Auto 


Escape  plans  are  usually  well  rehearsed 
and  executed. 

-  Many  terrorists  want  to  survive  the  operation 
and  escape. 

-  Exception  is  suicide  operation  but  support 
personnel  may  need  to  get-away 

Exploitation  is  the  primary  objective  of  the 
operation. 


Media  control  measures,  prepared  statements,  and  a  host  of  other  preparations  are  made  to 
effectively  exploit  a  successful  operation. 

These  will  be  timed  to  take  advantage  of  media  cycles  for  the  selected  target  audiences. 
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Slide  31  « 

Most  Common  Terrorist  Threats 

Duration:  00:00:32 

Advance  mode:  Auto 

Ujt  )  Most  Common  Terrorist  Threats 

•  Bombings  (Suicide)  •  Ambushes 

•  Arson  •  Armed  Raid 

•  Kidnapping/Extortion  •  Hostage  Taking/  Barricade 

•  Hijacking  Situation 

•  Skyjacking  (Suicide)  '  Mannings 

•  Assassination 

•  Maritime  operations  IpsSSC  jr? 

•  Standoff  attacks 

1-2-31 

1 .  First  teaching  point.  Bombings  are  without  a  doubt  the  most  prevalent  way  to  accomplish 
a  terrorist  act  and  will  remain  so  for  the  foreseeable  future. 

2.  Hostage  taking  incidents  demand  a  higher  degree  of  expertise  and  planning  and  are 
usually  done  only  by  groups  that  have  evolved  to  a  high  degree  of  proficiency  in  other 
less  sophisticated  terrorist  acts.  We  will  discuss  all  these  common  terrorist  acts  in 
greater  detail  shortly. 

3.  IED 

Slide32 

Choices 

Interaction  type:  Choices 

Passing  score:  60 

Instructions:  None 

Custom  Message:  None 

A 

Learning  Game  Placeholder 

Learning  Game:  Choices 

Title:  Choices 

) 

1.  In  this  phase,  the  collection  of  information  on  a  large  number  of  potential  targets,  some  of 
which  may  never  be  attacked,  or  seriously  considered  for  attack. 

1)  Intelligence  and  surveillance 

2)  Target  selection  (Correct) 

3)  Pre-attack 

4)  Attack  rehearsal 

2.  During  this  phase  security  studies  and  design  escape  routes  are  conducted. 

1)  Attack  phase 

2)  Pre-attack  surveillance  and  planning  (Correct) 

3)  Target  selection 

4)  Delivery 

3.  Most  prevalent  way  to  accomplish  a  terrorist  act  is: 

1)  IED  bomb  (Correct) 

2)  shoe  bomb 

3)  plane  bomb 

4)  kidnapping 
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Slide  33 

Duration:  00:00:34 
Advance  mode:  Auto 


Future  Trends 


"Between  now  and  2015  terrorist  tactics  will  become 
increasingly  sophisticated  and  designed  to  achieve 
mass  casualties.” 


National  Intelligence  Council 

"Global  Trends  2015:  A  Dialogue  About  the  Future  With  Nongovernmental 
Experts"  Report  (Dec  2000). 


Terrorism  as  a  conflict  method  that  has  survived  and  evolved  through  several  millennia  to  flourish 
in  the  modern  information  age,  terrorism  continues  to  adapt  to  meet  the  challenges  of  emerging 
forms  of  conflict,  and  exploit  developments  in  technology  and  society.  Terrorism  has 
demonstrated  increasing  abilities  to  adapt  to  counter-terrorism  measures  and  political  failure. 
Terrorist  are  developing  new  capabilities  of  attack  and  improving  the  efficiency  of  existing 
methods.  Additionally,  terrorist  groups  have  shown  significant  progress  in  escaping  from  a 
subordinate  role  in  nation-state  conflicts,  and  becoming  prominent  as  international  influences  in 
their  own  right.  They  are  becoming  more  integrated  with  other  sub-state  entities,  such  as  criminal 
organizations  and  legitimately  chartered  corporations,  and  are  gradually  assuming  a  measure  of 
control  and  identity  with  national  governments. 


No  notes  for  this  slide: 


Slide  35 

Slide  35 


END 


Duration:  00:00:05 
Advance  mode:  Auto 
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APPENDIX  E.  STORYBOARD  FOR  LESSON  TWO. 
INSTRUCTIONAL  MATERIALS 
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Slide  1  ® 

Slide  1 

Duration:  00:00:06 

Advance  mode:  Auto 

MOVES  INSTITUTE 

1/ ■  ~ 

IMIII 

Defense  in  Depth  / Preplanned 
Responses 

Lesson  Topic 

2.1 

No  notes  for  this  slide 

Slide  2  ® 

Enabling  Objectives 

Duration:  00:00:30 

Advance  mode:  Auto 

rvflia  Enabling  Objectives 

•  STATE  the  purpose  of  Defense  in  Depth 

•  DESCRIBE  the  principles  of  Defense  in 

Depth 

•  DESCRIBE  the  Assess,  Warning  and 

Threat  Zones  as  used  in  the  antiterrorism 
program  concept  of  Defense  in  Depth 

•  DESCRIBE  a  Naval  Vessel  Protection 

Zone  (NVPZ) 

No  notes  for  this  slide 

Slide  3  0 

Enabling  Objectives 

Duration:  00:00:13 

Advance  mode:  Auto 

pv/iiii  Enabling  Objectives 

•  DESCRIBE  the  purpose  of  pre¬ 
planned  responses. 

•  DESCRIBE  the  likely  terrorist  threats 
for  developing  pre-planned 
responses. 

No  notes  for  this  slide 
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Slide  4  ® 

Introduction 

Duration:  00:00:22 
Advance  mode:  Auto 


Introduction 


“AT/FP  training 
must  be  elevated  to 
the  same  priority  as 
primary  mission 
training.” 


“OPERATIONALIZE  AT/FP!!!!!!” 


As  a  result  of  the  commission  on  the  investigation  of  the  USS  Cole  attack  it  was  recommended 
that  “AT/FP  training  must  be  elevated  to  the  same  priority  as  primary  mission  training.” 

”  We  need  to  make  AT/FP  as  important  as  damage  control,  Anti-Air  warfare  and  other  Navy 
primary  missions. 


Slide  5  « 

Defense  in  Depth 

Duration:  00:00:16 
Advance  mode:  Auto 


Defense  in  Depth 


Purpose:  To  provide  principles  of 
Defense  in  Depth  and  associated 
concepts  of  the  Navy  Antiterrorism 
Program  in  order  to  link  antiterrorism 
countermeasures  with  specified  tasks. 


Defense  in  Depth 
a.  Purpose 

(1 )  To  provide  principles  of  Defense  in  Depth  and  associated  concepts  of  the  Navy  Anti- 
terrorism  Program  in  order  to  link  anti-terrorism  countermeasures  with  specified  tasks. 

Slide  6  « 

Defense  in  Depth 

Duration:  00:00:24 
Advance  mode:  Auto 


Defense  in  Depth 


Engaging  the  enemy  at  the  earliest 
opportunity  with  security  forces. 


•  Employing  weapons  at  maximum 
effective  range. 


•  Using  blocking  positions,  obstacles,  and 
supplementary  positions  throughout  the 
area. 


•  Positioning  and  moving  reserves  and  fire 
support  units  (Reaction  Force). 


Conventional  operations  Defense  in  Depth  is  achieved  by: 

(1 )  Engaging  the  enemy  at  the  earliest  opportunity  with  security  forces. 

(2)  Employing  weapons  at  maximum  effective  range 

(3)  Using  blocking  positions,  obstacles,  and  supplementary  positions  throughout  the  area. 
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(4)  Positioning  and  moving  reserves  and  fire  support  units  (Reaction  Force). 


Slide  7  ® 

Principles  -  Layered  Defense 

Duration:  00:00:08 
Advance  mode:  Auto 


Principles  -  Layered 
Defense 

Three  Zones: 


-  Assessment  (Detect) 


-  Warning  (Deter) 


-  Threat  (Defend/Mitigate) 


The  three  principles  of  Layered  defense 

(1)  Assessment 

(2)  Warning 

(3)  Threat 


Slide  8  ® 

Defense  in  Depth 

Duration:  00:00:28 
Advance  mode:  Auto 


Defense  in  Depth 

Assessment  Zone  -  allows  for  initial  ID  of  potential 
threat. 


“We  must  separate  the  terrorists  from  the 
background  noise.  ”  ADM  Gehman 


Warning  Zone  -  provides  the  area  where  the 
security  force  may  respond  with  initial  and 
escalating  measures  including  warnings, 
challenges,  use  of  non-lethal  weapons  if 
appropriate,  and  if  time  permits  shouldering  the 
threat. 

Threat  Zone  -  engagement  and  destruction  of  the 
threat 


Assessment  Zone 

(a)  Allows  for  initial  ID  of  potential  threat. 

Warning  Zone 

(a)  Provides  the  area  where  the  security  force  may  respond  with  initial  and  escalating  measures 
including  warnings,  challenges,  use  of  non-lethal  weapons  if  appropriate,  and  if  time  permits 
shouldering  the  threat. 

Threat  Zone 

(a)  Engagement  and  destruction  of  the  threat 
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Slide  9  ® 

Three  Zones 

Duration:  00:00:12 
Advance  mode:  Auto 


ID,  Track,  Assess,  and  Neutralize 


“Set  Conditions  for  the  Use  Of  Force” 


The  purposes  of  the  three  zones  are  to: 
“ID,  Track,  Assess,  Neutralize” 

“Set  Conditions  for  Use  of  Force” 


Slide  10 

NVPZ 

Duration:  00:01:16 
Advance  mode:  Auto 


The  Naval  Vessel  Protection  Zone  uses  the  same  principles  of  layered  defense. 

The  assessment  zone,  in  the  assessment  zone  contacts  are  identified  by  means  of  radar,  radio 
communications  or  visual. 

The  warning  zone,  in  the  warning  zone  contacts  are  given  warnings  and  challenges. 

(a)  Warnings  are  conducted  in  English  and  in  the  local  language  if  in  a  foreign  country. 

(b)  Challenges  are  conducted  in  English  and  in  the  local  language  if  in  a  foreign  country. 

(c)  Ensure  that  the  vessel  being  challenge  is  aware  that  he  is  being  challenged  by  Non-Lethal 
means. 

The  threat  zone,  if  the  contact  does  not  display  compliance  with  the  previous  warnings  the  watch 
stander  has  to  determine  the  threat  of  the  contact. 

Determining  the  threat  is  done  by  determining  the  contacts  capability,  opportunity  and  intent. 
Capability  refers  to:  Does  the  contact  posses  the  capability  to  harm  our  forces? 

Opportunity  refers  to:  Is  there  an  opportunity  for  the  contact  to  harm  our  forces? 

Intent  refers  to:  Is  the  intent  of  the  contact  to  harm  our  forces? 
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Slide  11  ® 

Inport  Security 

Duration:  00:01:12 
Advance  mode:  Auto 


Observation 

-  Small  Boats 

-  Sea  Barriers 

-  Personal  Arms 


i  External  Security 


-  Armed  Sentries 

-  SMGs 

-  Riot  Control 
Agents 

-  Hardened 
Barricades 

- 100%  Illumination 
-Armed  Sentries 


The  Naval  Vessel  Protection  Zone  and  the  Defense  in  depth  concept  also  applies  to  ships  in-port 
The  Extended  security  zone  -  delineated  by  the  Dashed  Blue  line  corresponds  to  the  assessment 
zone. 

Assets  used  in  this  area  are: 

(a)  Shore  side: 

HMMVs  or  patrol  cars 

Armed  Personnel  with  Id  checks  at  the  gates. 

(b)  Sea  side 
Communications 
Radar  equipment 
Small  Boats 

The  Perimeter  security  zone  -  delineated  by  the  Solid  green  corresponds  to  the  warning  zone. 
Assets  used  in  this  area  are: 

(a)  Shore  side 

Armed  Pier  sentries  at  the  ECPs 

(b)  Sea  side 
Small  Boats 
Sea  Barriers 

The  External  security  zone  -  delineated  by  the  solid  yellow  line  corresponds  to  the  threat  zone. 
Assets  used  in  this  area  are: 

(a)  Shore  side 
Armed  Pier  rovers 
Armed  Topside  rovers 

100%  illumination  of  pier  areas 

(b)  Sea  side 
Small  Boats 
Sea  Barriers 

Armed  Topside  rovers 

100%  illumination  of  ship  boundaries  with  the  water _ 
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Slide  12 

Duration:  00:00:29 
Advance  mode:  Auto 


High  Value  Assets 

High  value  assets  are  those  assets  that  the  enemy  wants  to  target  because: 

The  target’s  symbolic  value 

The  target’s  potential  to  generate  high  profile  media  attention 
The  target  is  the  objective  of  the  terrorist  group 

This  picture  was  taken  prior  to  the  USS  Cole  attack,  security  was  not  our  number  one  priority. 


Slide  13 

Slide  13 

Duration:  00:00:10 
Advance  mode:  Auto 


The  Naval  Vessel  Protection  Zone  and  the  Defense  in  depth  concept  also  applies  to  ships 
transiting  in  or  out  of  port  and  in  narrow  channels. 


Slide  14 

Pre-Planned  Reponses  (PPR) 

Duration:  00:00:24 
Advance  mode:  Auto 


V. 


Pre-Planned  Reponses 
(PPR) 

PPRs  are  developed,  exercised  actions 
and  measures  that  are  implemented  to: 

-  Track 

-  Assess 

-  Neutralize  terrorist  threats 

PPRs  are  the  tactics,  techniques  used 
to  carry  out  the  AT  program  concept  of: 

-  Detect 

-  Deter 

-  Defend 

-  Mitigate 


Pre-Planned  Reponses  (PPR’s)  are  developed,  exercised  actions  and  measures  that  are 
implemented  to: 

a.  Track 

b.  Assess 
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c.  Neutralize  terrorist  threats 

PPR’s  are  the  tactics,  techniques  used  to  carry  out  the  AT  program  concept  of: 

(1 )  Detect 

(2)  Deter 

(3)  Defend 


(4)  Mitigate 

Slide  15® 

Pre-Planned  Reponses  (PPR) 

Duration:  00:00:31 

Advance  mode:  Auto 

Pre-Planned  Reponses 
*  ^  (PPR) 

•  Likely  threats: 

-  Small  boats/Deep  draft  vessels 

-  Swimmers/Mines 

-  Personnel-borne  lEDs 

-  Vehicle-born  lEDs 

-  Rail-born  lEDs 

-  Aircraft 

-  Standoff  attacks 

-  Man-portable  defense  systems 
(MANPADS) 

-  CBNE  attacks  (covert-overt) 

Likely  threats  to  U.S.  Assets 

(1 )  Small  boats/Deep  draft  vessels 

(2)  Swimmers/Mines 

(3)  Personnel  borne  lEDs 

(4)  Vehicle-born  lEDs 

(5)  Rail-born  lEDs 

(6)  Aircraft 

(7)  Standoff  attacks 

(8)  Man-portable  defense  systems  (MANPADS) 

(9)  CBRNE  attacks  (covert-overt) 

Slide  16® 

Pre-Planned  Reponses  (PPR) 

Duration:  00:00:23 

Advance  mode:  Auto 

HHI  Pre-Planned  Reponses 
(PPR) 

•  PPRs  are  inherent  in  your  COA 
development 

-  During  war  gaming  each  friendly  COA  is 
examined  against  selected  threat  COAs 

-  If  a  certain  threat  attacks  or  endangers  the 
command,  it  is  essential  to  know  how  the 
command  will  react  with  PPRs. 

•  Sufficient 

•  Timely 

PPR’s  are  inherent  in  your  course  of  action  (COA)  Development 

(1)  During  war  gaming  each  friendly  COA  is  examined  against  selected  threat  COAs 

(2)  If  a  certain  threat  attacks  or  endangers  the  command,  it  is  essential  to  know  how  the 
command  will  react  with  PPR’s 

(a)  Sufficient 

(b)  Timely _ 
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Pre-Planned  Reponses 
(PPR) 

Pre-Planned  Responses 

-  NTTP  3-07.2.1,  Chapter  6 

-  Good  check  list  of  PPRs 

-  Must  look  beyond  check  lists 

Must  think  critically  and  dynamically 
about  potential  threats  for  each 
mission  and  for  each  vulnerability. 


Pre-Planned  Responses 

(1)  NTTP  3-07.2.1,  Rev  A,  Chapter  6 

(2)  Good  check  list  of  PPR’s 

(3)  Must  look  beyond  check  lists 

(4)  Must  think  critically  and  dynamically  about  potential  threats  for  each  mission  and  for  each 


vulnerability 

Slide  18  f> 

Set  up  defense  in  depth  for  the 

NAVSTA  scenario  and  list  and 

observe  the  pre-planned  responses 

Set  up  defense  in  depth 

for  boat-born  IED. 

for  the  NAVSTA  scenario 

Duration:  00:00:13 

and  list  and  observe  the 

Advance  mode:  Auto 

pre-planned  responses 
for  boat-born  IED. 

Slide  17 

Pre-Planned  Reponses  (PPR) 


Duration:  00:00:26 
Advance  mode:  Auto 
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APPENDIX  F.  STUDY  I  PRE-TEST  WITH  ANSWERS 


NAME 


Pre-Test: 


Tme/False,  5  points,  1  attempt(s)  permitted. 

1.  In  accordance  to  the  Department  of  Defense  terrorism  is  defined  as: 

The  calculated  use  of  violence  or  threat  of  violence  to  inculcate  fear;  intended  to 
coerce  or  to  intimidate  governments  or  societies  in  the  pursuit  of  goals  that  are 
generally  political,  religious  or  ideological”. 

(In  accordance  to  the  Department  of  Defense  terrorism  is  defined  as:  The 
calculated  use  of  violence  or  threat  of  violence  to  inculcate  fear;  intended  to  coerce 
or  to  intimidate  governments  or  societies  in  the  pursuit  of  goals  that  are  generally 
political,  religious  or  ideological”.  ) 


Correct 

Choice 

X 

True 

False 

True/False,  5  points,  1  attempt(s)  permitted. 

2.  The  categories  of  terrorist  organizations  are: 

government  affiliated,  motivation,  ideological  (The  categories  of  terrorist 
organizations  are:  government  affiliated,  motivation,  ideological) 


Correct 

Choice 

X 

True 

False 

Multiple  Response,  5  points,  1  attempt(s)  permitted. 

3.  Multiple  response  question: 

Government  affiliated  terrorist  organization  are  comprised  of  the  following: 
(Multiple  response  question:  Government  affiliated  terrorist  organization  are 
comprised  of  the  following:) 


Correct 

Choice 

X 

non-state  supported 

1  Religious  | 

X 

state-supported 

1  Government  j 

X 

state-directed 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 
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4.  Motivation  affiliated  terrorist  organization  are  comprised  of  the  following  except: 


Correct  Choice _ 

_ Separatist _ 

_ ethnocentric  groups 

X  Ideological _ 

_ Nationalistic _ 

_ Revolutionary 


Multiple  Response,  5  points,  1  attempt(s)  permitted. 

5.  Multiple  response  question: 

Ideological  affiliated  terrorist  organization  are  comprised  of  the  following  except: 
(Multiple  response  question:  Ideological  affiliated  terrorist  organization  are 
comprised  of  the  following  except:) 


Correct  Choice 

Political 

X  Nationalist 

Religious 

social  orientation 

X  Terrorism 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

6.  Political  affiliated  terrorist  organization  are  comprised  of  the  following: 

Correct  Choice 

X  Right  wing,  Left  wing,  Anarchist 

Right  wing,  Left  wing,  Center 

Center,  Left  wing,  Anarchist 

Left  wing,  Anarchist,  Center 

Right  wing,  Center,  Anarchist 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

7.  Terrorist  hope  to  accomplish  the  following  goals  except: 

Correct  Choice 

Recognition 

Vengeance 

Weaken  government 

X  create  their  own  government 

discourage  foreign  investment 
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True/False,  5  points,  1  attempt(s)  permitted. 

8.  The  definition  of  internal  terrorist  training  is:  conducted  at  the  cell  level  by 
members  of  the  group  within  the  target  country,  usually  in  a  clandestine  fashion. 
(The  definition  of  internal  terrorist  training  is:  conducted  at  the  cell  level  by 
members  of  the  group  within  the  target  country,  usually  in  a  clandestine  fashion.) 


Correct 

Choice 

X 

True 

False 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

9.  These  training  programs  are  included  in  a  typical  terrorist  training  program 
except: 

(These  training  programs  are  included  in  a  typical  terrorist  training  program 
except:  ) 


Correct  Choice 

Physical  condition 

Weapons  training 

Political  indoctrination 

Combat  tactics 

X  Capital  investment 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

10.  Which  one  of  these  is  a  characteristic  of  a  terrorist  group: 

Correct  Choice 

Overt 

Military  strong 

Based  outside  urban  areas 

Poor  training 

X  Highly  mobile 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

11.  The  first  stage  of  a  terrorist  attack  is: 

Correct  Choice 

X  Target  selection 

Intelligence  and  surveillance 

Pre-attack 

Specific  target  selection 

Attack  rehearsal 
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Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

12.  In  this  phase,  Targets  showing  potential  vulnerabilities  are  given  a  higher 
priority  of  effort.  This  priority  establishes  the  requirement  to  gather  additional 
information  on  the  targets’  patterns  over  time. 


Correct  Choice 

Target  selection 

X  Intelligence  and  surveillance 

Pre-attack 

Specific  target  selection 

Attack  rehearsal 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

13.  Once  a  target  is  selected  the  following  surveillance  is  conducted  except: 

Correct  Choice 

Conduct  security  studies 

Recruit  specialized  operatives 

Design  and  test  escape  routes 

Decide  on  attack  weapon 

X  Determine  the  objective  of  the  group 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

14.  The  last  step  of  a  terrorist  attack  is: 

Correct  Choice 

X  Escape  and  exploitation 

Attack 

Actions  on  the  objective 

Attack  rehearsal 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

15.  All  of  these  are  common  terrorist  threats  except: 

Correct  Choice 

Bombing 

Arson 

Ambushes 

Maiming 

X  War 
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Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

16.  Based  on  what  we  learned  on  this  course  a  future  terrorist  tactic  may  include: 


Correct  Choice 

Computer  attack 

Bombs  delivered  by  animals 

Shoe  bombs 

X  CBRNE 

Liquid  bombs  on  airplanes 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

17.  Terrorist  will  be  looking  to  achieve  this  type  of  casualty  by  the  year  2015: 

Correct  Choice 

Network  casualties 

Economy  casualties 

X  Mass  casualties 

Political  casualties 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

18.  These  are  tactical  advantages  of  a  terrorist  conducting  an  attack  except: 

Correct  Choice 

Choice  of  time 

Choice  of  place 

X  Numbers  of  security  forces 

Surprise 

Diversions  for  follow  up  attacks 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

19.  These  type  of  groups  are  dedicated  to  the  overthrow  of  an  established  order  and 
replacing  it  with  a  new  political  or  social  structure.  (These  type  of  groups  are 
dedicated  to  the  overthrow  of  an  established  order  and  replacing  it  with  a  new 
political  or  social  structure.) 


Correct  Choice _ 

_ Separatist  group 

_ Ethnocentric  group 

X  Revolutionary  group 

_ Political  group _ 
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Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

20.  These  type  of  groups  operate  as  an  agent  of  a  government  and  receive 
substantial 

intelligence,  logistic,  and  operational  support  from  the  sponsoring  government. 
(These  type  of  groups  operate  as  an  agent  of  a  government  and  receive  substantial 
intelligence,  logistic,  and  operational  support  from  the  sponsoring  government.  ) 
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APPENDIX  G.  STUDY  I  POST-TEST  WITH  ANSWERS 


NAME _  Pre-Test: 

Tme/False,  5  points,  1  attempt(s)  permitted. 

1.  The  following  elements  are  essential  in  the  definition  of  terrorism  in  accordance 
to  the  Department  of  Defense: 

violence  or  threat  of  violence,  fear,  coercion,  pursuit  of  goals  (The  following 
elements  are  essential  in  the  definition  of  terrorism  in  accordance  to  the  Department 
of  Defense:  violence  or  threat  of  violence,  fear,  coercion,  pursuit  of  goals) 


Correct 

Choice 

X 

True 

False 

Multiple  Response,  5  points,  1  attempt(s)  permitted. 

2,  Multiple  response  question: 

Which  is/are  categories  of  terrorist  organization?  (Multiple  response  question: 
Which  is/are  categories  of  terrorist  organization?) 


Correct 

Choice 

X 

government  affiliated 

state  governed 

X 

Motivation 

1  Separatist  | 

X 

Ideological 

Multiple  Response,  5  points,  1  attempt(s)  permitted. 

3.  Which  is  an  indication  of  a  government  affiliated  terrorist  organization? 


Correct  Choice 


X  state  support 


Violence 


Threat 


Fear 


Coercion 
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Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

4.  These  are  included  in  a  government  affiliated  terrorist  organization  except: 


Correct  Choice 

Separatist 

ethnocentric  groups 

Nationalistic 

Revolutionary 

X  Political 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

5.  Which  is  included  in  a  ideological  affiliated  terrorist  organization? 

Correct  Choice 

X  Religious 

Rationalist 

Nationalist 

Anarchist 

Multiple  Response,  5  points,  1  attempt(s)  permitted. 

6.  Multiple  response  question: 

Which  is/are  included  in  a  political  affiliated  terrorist  organization?  (Multiple 
response  question:  Which  is/are  included  in  a  political  affiliated  terrorist 
organization?) 


Correct 

Choice 

X 

right  wing 

X 

left  wing 

X 

Anarchist 

Religious 

Radicalism 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

7.  Which  one  of  these  is  a  goal  terrorists  do  not  want  to  achieve? 


Correct  Choice _ 

Eliminate  western  influence,  recognition 

_ Free  prisoner,  satisfy  vengeance _ 

_ Influence  government  decisions,  disrupt  government 

X  Generate  money,  create  their  own  government _ 
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Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

8.  Which  2  training  programs  are  not  in  a  typical  terrorist  training  program. 
(Which  2  training  programs  are  not  in  a  typical  terrorist  training  program.  ) 


Correct  Choice _ 

_ Physical  condition,  survival  skills _ 

_ Communications,  political  indoctrination 

_ Weapons,  explosive  training _ 

X  Electronic,  network  warfare _ 


True/False,  5  points,  1  attempt(s)  permitted. 

9.  In  accordance  with  the  DoD  the  definition  of  external  terrorist  is: 

Non-group  members  outside  the  operational  area  conduct  external  training.  It  can 
take  two  different  forms.  First,  the  operatives  may  travel  abroad,  individually  or  as 
a  group,  to  training  camps  or  to  other  permanent  institutions  for  their  training. 

(In  accordance  with  the  DoD  the  definition  of  external  terrorist  is:  Non-group 
members  outside  the  operational  area  conduct  external  training.  It  can  take  two 
different  forms.  First,  the  operatives  may  travel  abroad,  individually  or  as  a  group, 
to  training  camps  or  to  other  permanent  institutions  for  their  training.  ) 


Correct 

Choice 

X 

True 

False 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

10.  Which  one  of  these  is  a  characteristic  of  a  terrorist  group: 


Correct  Choice 


X 


Covert 


Urban  based 


Well  trained 


Overt 


Military  weaker 


Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

11.  In  this  phase,  the  collection  of  information  on  a  large  number  of  potential 
targets,  some  of  which  may  never  be  attacked,  or  seriously  considered  for  attack. 


Correct  Choice _ 

X  Target  selection _ 

_ Intelligence  and  surveillance 

Pre-attack 

_ Specific  target  selection 

Attack  rehearsal 
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Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

12.  Selection  of  a  target  in  the  target  selection  stage,  for  actual  operational  planning 
considers  some  of  the  following  factors  except: 


Correct  Choice 

Will  the  target  attract  high  profile  media  attention? 

Does  success  make  the  desired  statement  to  the  correct  target  audience(s)? 

Is  the  effect  consistent  with  objectives  of  the  group? 

the  target  provide  an  advantage  to  the  group  by  demonstrating  its  capabilities? 

X  Are  escape  routes  available? 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

13.  During  this  phase  security  studies  and  design  escape  routes  are  conducted. 

Correct  Choice 

Attack  phase 

X  Pre-attack  surveillance  and  planning 

Target  selection 

Attack 

Attack  rehearsal 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

14.  Which  one  of  these  is  not  a  stage  of  the  terrorist  planning  cycle. 

Correct  Choice 

Target  selection 

Intelligence  and  surveillance 

Attack 

Actions  on  the  objective 

X  Communication  with  media 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

15.  Most  prevalent  way  to  accomplish  a  terrorist  act  is: 

Correct  Choice 

Kidnapping 

X  IED  bombings 

Carjacking 

Plane  hijacking 

Arson 
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Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

16.  The  following  are  future  trends  of  terrorist  tactics  except: 


Correct  Choice 

CBRNE 

IED  bombs 

X  UAV  delivered  bombs 

High  yield  explosives 

Nuclear  dirty  bomb 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

17.  The  primary  objective  of  a  terrorist  plan  is: 

Correct  Choice 

X  Exploitation 

Economic  success 

People  anger 

Suicide  for  afterlife  rewards 

Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

18.  These  group  objective  is  the  separation  from  existing  entities  through 
independence,  political  autonomy,  or  religious  freedom  or  domination.  Their 
ideologies  subscribe  to  include  social  justice  or  equity,  anti-imperialism,  as 
well  as  the  resistance  to  conquest  or  occupation  by  a  foreign  power. 

(These  group  objective  is  the  separation  from  existing  entities  through 
independence,  political  autonomy,  or  religious  freedom  or  domination.  Their 
ideologies  subscribe  to  include  social  justice  or  equity,  anti-imperialism,  as  well  as 
the  resistance  to  conquest  or  occupation  by  a  foreign  power.  ) 


Correct  Choice _ 

X  Separatist  group 

_ Ethnocentric  group 

_ Revolutionary  group 

_ Political  group _ 
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Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

19.  These  are  terrorist  groups  that  operate  autonomously,  receiving  no 
significant  support  from  any  government. 

(These  are  terrorist  groups  that  operate  autonomously,  receiving  no  significant 
support  from  any  government.  ) 


Correct  Choice 


State  supported 


X  Non-state  supported 


State  directed 


Proxy  army 


Multiple  Choice,  5  points,  1  attempt(s)  permitted. 

20.  This  is  the  definition  of  a  deliberate  terrorist  element. 


Correct  Choice _ 

X  Terrorism  is  an  activity  planned  and  intended  to  achieve  particular  goals. 

_ Terrorist  groups  demand  change,  revolution  or  political  movement. _ 

_ The  intended  result  is  to  create  a  psychological  terror _ 

Violence,  coercion,  and  destruction  are  used  in  the  commission  of  the  act. 

Violence,  coercion,  and  destruction  are  used  in  the  commission  of  the  act. 
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APPENDIX  H.  STUDY  II  PRE-TEST  WITH  ANSWERS 


NAME _  Pre-Test: 

Pre-Test  objectives: 

1.  Measure  the  student’s  knowledge  about  the  following  subjects: 

•  Defense  in  Depth 

•  Assessment,  Warning  and  Threat  Zones  as  used  in  the  anti-terrorism  program 

•  Naval  Vessel  Protection  Zone  (NVPZ) 

•  Pre-planned  responses. 


1 .  What  is  the  purpose  of  defense  in  depth? 


a.  To  provide  the  ATO  with  pre-planned  responses  in  case  of  terrorist  attacks. 

X 

b.  To  provide  principles  of  Defense  in  Depth  and  associated  concepts  of  the  Navy 
Anti-terrorism  Program  in  order  to  link  anti-terrorism  countenneasures  with 
specified  tasks. 

c.  To  engage  the  enemy  at  the  earliest  opportunity. 

d.  Employ  weapons  at  the  maximum  effective  range. 

2.  Which  answer  contains  2  principles  of  defense  in  depth? 


a.  The  use  of  small  boats  and  shipboard  security  forces. 

b.  Engage  enemy  forces  at  the  earliest  opportunity  to  prevent  further  damage. 

X 

c.  Engaging  the  enemy  at  the  earliest  opportunity,  employ  weapons  at  maximum 
effective  range. 

d.  using  obstacles  throughout  the  area  and  small  boats. 

3.  What  are  the  principles  of  layered  defense  as  used  in 
Anti  Terrorism  Force  Protection  (AT/FP)? _ 


X 

a.  Assessment,  warning  and  threat. 

b.  Assessment,  warning  and  capability  opportunity  intent  (COI). 

c.  Assessment,  threat  and  COI. 

d.  none  of  the  above. 

4.  What  is  the  proper  definition  for  assessment  zone: 


X 

a.  Allows  for  initial  ID  of  potential  threat. 

b.  Provides  the  area  where  the  security  force  may  respond  with  initial  and 
escalating  measures  including  warnings,  challenges,  use  of  non-lethal  weapons  if 
appropriate,  and  if  time  pennits  shouldering  the  threat. 
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c.  Engagement  and  destruction  of  the  threat. 

d.  Area  where  capability,  opportunity  and  intent  are  determined. 

5.  What  is  the  proper  definition  for  warning  zone: 


a.  Allows  for  initial  ID  of  potential  threat. 

X 

b.  Provides  the  area  where  the  security  force  may  respond  with  initial  and 
escalating  measures  including  warnings,  challenges,  use  of  non-lethal  weapons  if 
appropriate,  and  if  time  pennits  shouldering  the  threat. 

c.  Engagement  and  destruction  of  the  threat. 

d.  Area  where  capability,  opportunity  and  intent  are  determined. 

6.  What  is  the  proper  definition  for  threat  zone: 


a.  Allows  for  initial  ID  of  potential  threat. 

b.  Provides  the  area  where  the  security  force  may  respond  with  initial  and 
escalating  measures  including  warnings,  challenges,  use  of  non-lethal  weapons  if 
appropriate,  and  if  time  pennits  shouldering  the  threat. 

X 

c.  Engagement  and  destruction  of  the  threat. 

d.  Area  where  capability,  opportunity  and  intent  are  determined. 

7.  What  is  the  purpose  of  the  Assessment,  warning  and  threat  zones? 


a.  Neutralize  terrorist  contacts. 

b.  Warn  unidentified  vessels  of  the  Naval  Vessel  Protection  Zone. 

X 

c.  ID,  Track,  Assess,  Neutralize  and  Set  Conditions  for  Use  of  Force. 

d.  Allows  for  initial  ID  of  potential  threat. 

8.  How  is  the  Naval  Vessel  Protection  Zone  organized? 


a.  ID 

b.  Warn 

c.  Assessment 

X 

d.  threat 

9.  What  asset  is  used  in  the  Assessment  zone? 


X 

a.  Patrol  craft. 

b.  Air  search  radars. 

c.  Intelligence  briefs. 

d.  NCIS 
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10.  Which  actions  the  watch-stander  must  take  when  an  unidentified  vessel  enters 
the  warning  zone? _ 


a.  Ensure  that  the  vessel  being  challenge  is  aware  that  he  is  being  challenged 

b.  Small  arm  fire  across  their  bow. 

c.  Use  the  patrol  boat  to  shoulder  the  contact. 

X 

d.  Verbal  Warnings  in  English  and  in  the  local  language  if  in  a  foreign  country. 

1 1 .  Which  criteria  the  watch  stander  uses  to  determine  threat? 


a.  Opportunity 

b.  Capability 

c.  Intent 

X 

d.  threat 

12.  What  is  the  purpose  of  Pre-planned  responses? 


a.  Rehearse  terrorist  attacks. 

X 

b.  Track,  assess  and  neutralize  terrorist  threats. 

c.  Plan  for  terrorist  possible  actions. 

d.  Neutralize  terrorist  threats. 

13.  Can  Pre-Planned  Responses  be  modified  to  tailor  specific  situations? 


X 

a.  True 

b.  False 

14.  Multiple  answer  question 

Select  (3)  likely  threats  that  any  command  has  to  be  prepared  for? 


X 

a.  Small  boats/Deep  draft  vessels 

X 

b.  Swimmers/Mines 

c.  submarine  attacks. 

d.  WMD. 

X 

e.  CBRNE  attacks  (covert-overt) 

15.  How  are  Pre-Planned  Responses  used  in  AT/FP  program? 


a.  Used  to  prevent  terrorist  attacks. 

b.  Used  in  after  action  reports  to  evaluate  incidents. 

X 

c.  During  war  gaming  each  friendly  COA  is  examined  against  selected  threat 
COAs. 

d.  check  list. 
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APPENDIX  I.  STUDY  II  POST-TEST  WITH  ANSWERS 


NAME _  Post-Test: 

Post-Test  objectives: 

1.  Measure  the  student’s  knowledge  about  the  following  subjects: 

•  Defense  in  Depth 

•  Assessment,  Warning  and  Threat  Zones  as  used  in  the  anti-terrorism  program 

•  Naval  Vessel  Protection  Zone  (NVPZ) 

•  Pre-planned  responses. 


1 .  What  is  the  purpose  of  defense  in  depth? 


a.  To  provide  the  ATO  with  pre-planned  responses  in  case  of  terrorist  attacks. 

X 

b.  To  provide  principles  of  Defense  in  Depth  and  associated  concepts  of  the  Navy 
Anti-terrorism  Program  in  order  to  link  anti-terrorism  countenneasures  with 
specified  tasks. 

c.  To  engage  the  enemy  at  the  earliest  opportunity. 

d.  Employ  weapons  at  the  maximum  effective  range. 

2.  Are  Pre-Planned  Responses  used  in  Course  of  Action  development? 


X 

a.  True 

b.  False 

3.  What  does  a  good  AT/FP  Officer  must  do  to  ensure  that  he  assesses  all  the 
possible  terrorist  threats? _ 


a.  Ask  a  lot  of  questions. 

b.  Use  check  lists  only. 

X 

c.  Must  think  critically  and  dynamically  about  potential  threats  for  each  mission 
and  for  each  vulnerability. 

d.  Practice,  practice,  practice. 

4.  Which  are  the  3  principles  of  layered  defense  as  used  in 
Anti  Terrorism  Force  Protection  (AT/FP)? _ 


X 

a.  Assessment,  warning  and  threat. 

b.  Assessment,  warning  and  capability  opportunity  intent  (COI). 

c.  Assessment,  threat  and  COI. 

d.  none  of  the  above. 
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5.  What  is  the  proper  definition  for  assessment  zone: 


X 

a.  Allows  for  initial  ID  of  potential  threat. 

b.  Provides  the  area  where  the  security  force  may  respond  with  initial  and 
escalating  measures  including  warnings,  challenges,  use  of  non-lethal  weapons  if 
appropriate,  and  if  time  pennits  shouldering  the  threat. 

c.  Engagement  and  destruction  of  the  threat. 

d.  Area  where  capability,  opportunity  and  intent  are  determined. 

6.  What  is  the  proper  definition  for  warning  zone; 


a.  Allows  for  initial  ID  of  potential  threat. 

X 

b.  Provides  the  area  where  the  security  force  may  respond  with  initial  and 
escalating  measures  including  warnings,  challenges,  use  of  non-lethal  weapons  if 
appropriate,  and  if  time  pennits  shouldering  the  threat. 

c.  Engagement  and  destruction  of  the  threat. 

d.  Area  where  capability,  opportunity  and  intent  are  determined. 

7.  What  is  the  proper  definition  for  threat  zone: 


a.  Allows  for  initial  ID  of  potential  threat. 

b.  Provides  the  area  where  the  security  force  may  respond  with  initial  and 
escalating  measures  including  warnings,  challenges,  use  of  non-lethal  weapons  if 
appropriate,  and  if  time  pennits  shouldering  the  threat. 

X 

c.  Engagement  and  destruction  of  the  threat. 

d.  Area  where  capability,  opportunity  and  intent  are  determined. 

8.  What  is  the  purpose  of  the  assessment,  warning  and  threat  zones? 


a.  Neutralize  terrorist  contacts. 

b.  Warn  unidentified  vessels  of  the  Naval  Vessel  Protection  Zone. 

X 

c.  ID,  Track,  Assess,  Neutralize  and  Set  Conditions  for  Use  of  Force. 

d.  Allows  for  initial  ID  of  potential  threat. 

9.  How  is  the  Naval  Vessel  Protection  Zone  organized? 


a.  ID 

b.  Warn 

c.  Assessment 

X 

d.  Assessment 
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10.  What  asset  is  used  in  the  Assessment  zone? 


X 

a.  Patrol  craft. 

b.  Air  search  radars. 

c.  Intelligence  briefs. 

d.  NCIS 

1 1 .  Which  criteria  does  the  watch-stander  uses  to  determine  threat? 


a.  Opportunity 

b.  Capability 

c.  Intent 

X 

d.  HYU 

12.  What  is  the  purpose  of  Pre-planned  responses? 


a.  Rehearse  terrorist  attacks. 

X 

b.  Track,  assess  and  neutralize  terrorist  threats. 

c.  Plan  for  terrorist  possible  actions. 

d.  Neutralize  terrorist  threats. 

13.  Can  Pre-Planned  Responses  be  modified  to  tailor  specific  situations? 


X 

a.  True 

b.  False 

14.  Multiple  answer  question 

Select  (3)  likely  threats  that  any  command  has  to  be  prepared  for? 


X 

a.  Small  boats/Deep  draft  vessels 

X 

b.  Swimmers/Mines 

c.  submarine  attacks. 

d.  WMD. 

X 

e.  CBRNE  attacks  (covert-overt) 

15.  How  are  Pre-Planned  Responses  used  in  AT/FP  program? 


a.  Used  to  prevent  terrorist  attacks. 

b.  Used  in  after  action  reports  to  evaluate  incidents. 

X 

c.  During  war  gaming  each  friendly  COA  is  examined  against  selected  threat 
COAs. 

d.  check  list. 
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APPENDIX  J.  SAMPLE  PORT  BRIEFING  INFORMATION 

SHEET 


INFORMATION  SHEET  1 
TASK  3 

10  May  2007. 


From:  Commanding  Officer 


To:  Designated  Anti-terrorism  Officer 


Subj :  IN  PORT  SECURITY  PLAN  ICO  ANY  PORT  FOR  A  PORT  VISIT 


1.  Two  days  from  now,  at  approximately  2000  Local,  we  will  be  arriving  in  any  port  for  a  (4)  day  port  visit.  We  will  depart  at 
0600  Local  on  the  fourth  day. 


2.  The  Freedom  Fighters  United  (FFU)  is  an  Islamic  fundamentalist  based  group  consisting  of  approximately  10  members  with 
ties  to  Hizballah  and  HAMAS  terrorist  organizations  operating  in  the  Middle  East.  FFU  conducts  their  operations  covertly 
primarily  in  and  around  the  city  of  nay  city,  any  state.  Operatives  have  been  observed  in  the  past  around  the  area  of  NAVSTA 
nay  NAVSTA  to  what  appears  conducting  surveillance.  With  the  exception  of  protests  at  the  base  gates  in  April  2007  by 
members  of  the  FFU,  there  are  no  indications  of  preparations  for  an  attack  by  any  organization  to  military  assets  on  the  base.  FBI 
reports  that  the  FFU  have  recently  purchased  2  small  boats.  Last  week  an  Army  explosives  DEPOT  was  broken  into  and  large 
quantities  of  explosives  were  stolen.  FBI  is  investigating  but  preliminary  reports  point  to  the  FFU. 


3.  Any  NAVSTA  is  currently  in  FPCON  Bravo. 


4.  TASKING: 


a.  You  area  directed  to  review  directives  and  instructions  regarding  AT/FP  planning  and  In  Port  Security  Plans  (ISP). 


b.  Given  2  RHIBS,  pier  radar  and  barriers  use  the  AFTP  SavageStudio  to  develop  an  asset  deployment  and  use  given  pre 
planned  responses  against  explosive  laden  vessels. 
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APPENDIX  K.  INSTITUIONAL  REVIEW  BOARD  (IRB)  FORMS 


Naval  Postgraduate  School 

Informed  Consent  Form 

Introduction.  You  are  invited  to  participate  in  a  study  entitled  Training  effectiveness  of 
Force  Protection  Training  using  Computer-based  Instruction  with  embedded  Simulation 
being  conducted  by  the  Naval  Postgraduate  School  MOVES  Institute. 

Procedures.  Y ou  will  be  asked  to  complete  a  questionnaire  with  questions  about  your 
rank,  rate  and  assignments  previous  and  current.  You  will  conduct  a  pre-test  about  Force 
protection  knowledge,  that  way  we  can  measure  your  base  knowledge.  Second  you  will 
complete  a  CBT  lesson  on  Force  Protection,  and  then  you  will  complete  a  Force 
Protection  scenario.  The  conclusion  of  the  research  will  be  after  completion  of  the  post¬ 
test,  to  measure  if  any  knowledge  has  been  acquired.  Total  time  required  45  minutes. 

Risks  and  Benefits.  I  understand  that  this  project  does  not  involve  greater  than  minimal 
risk  and  involves  no  known  reasonably  foreseeable  risks  or  hazards  greater  than  those 
encountered  in  everyday  life.  I  have  also  been  informed  of  any  benefits  to  myself  or  to 
others  that  may  reasonably  be  expected  as  a  result  of  this  research. 

Compensation.  I  understand  that  no  tangible  compensation  will  be  given.  I  understand 
that  a  copy  of  the  research  results  will  be  available  at  the  conclusion  of  the  experiment  via 
e-mail  and  thesis  results  will  be  available  at  the  library. 

Confidentiality  &  Privacy  Act.  I  understand  that  all  records  of  this  study  will  be  kept 
confidential  and  that  my  privacy  will  be  safeguarded.  No  information  will  be  publicly 
accessible  which  could  identify  me  as  a  participant.  I  will  be  identified  only  as  a  code 
number  on  all  research  forms/data  bases.  My  name  on  any  signed  document  will  not  be 
paired  with  my  code  number  in  order  to  protect  my  identity.  I  understand  that  records  of 
my  participation  will  be  maintained  by  NPS  for  three  years,  after  which  they  will  be 
destroyed. 

Voluntary  Nature  of  the  Study.  I  understand  that  my  participation  is  strictly  voluntary, 
and  if  I  agree  to  participate,  I  am  free  to  withdraw  at  any  time  without  prejudice. 

Points  of  Contact.  I  understand  that  if  I  have  any  questions  or  comments  regarding  this 
project  upon  the  completion  of  my  participation,  I  should  contact  the  Principal  Investigator, 
,  Dr.  Don  Brutzman,  656-2149,  brutzman@nps.edu.  Any  medical  questions  should  be 
addressed  to  LTC  Eric  Morgan,  MC,  USA,  (CO,  POM  Medical  Clinic),  (831)  242-7550, 
eric.morgan@nw.amedd.army.mil.  Any  other  questions  or  concerns  may  be  addressed  to 
the  IRB  Chair,  LT  Brent  Olde,  656-3807,  baolde@nps.edu. 
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Statement  of  Consent.  I  have  been  provided  with  a  full  explanation  of  the  purpose, 
procedures,  and  duration  of  my  participation  in  this  research  project.  I  understand  how 
my  identification  will  be  safeguarded  and  have  had  all  my  questions  answered.  I  have 
been  provided  a  copy  of  this  form  for  my  records  and  I  agree  to  participate  in  this  study.  I 
understand  that  by  agreeing  to  participate  in  this  research  and  signing  this  form,  I  do  not 
waive  any  of  my  legal  rights. 


Participant’s  Signature 


Date 


Researcher’s  Signature 


Date 


138 


APPENDIX  L.  STUDENT  DEMOGRAPHICS  QUESTIONNAIRE 


NAVAL  POSTGRADUATE  SCHOOL,  MONTEREY,  CA  93943 

Transfer  of  Training  Study  between  Classroom-Based  and  Computer-Based 

Training 

Questionnaire: 

1.  Student  demographics: 

a.  What  is  your  rank? 

b.  What  is  your  rate  if  applicable? 

c.  gender 

d.  age 

e.  History  of  assignments: 

f.  What  is  your  total  time  in  the  Navy? 

g.  Do  you  have  any  previous  knowledge  of  ATFP  or  attended  any  other  ATFP  courses? 

h.  In  a  scale  of  1-5.  1.  no  knowledge  at  all  2.  little  knowledge  3.  basic  knowledge  4.  reasonable 
knowledge  5. expert  knowledge.  How  do  you  rate  your  ATFP  knowledge? 

i.  How  often  does  your  command  conduct  ATFP  training? 
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APPENDIX  M.  SAVAGESTUDIO  EXERCISE  STUDENT  TASK 

CHECKLIST 


SAVAGESTUDIO  EXERCISE  CHECKLIST 

Date: _  Student  #: _ 

Training  Objectives: 

1 .  Develop  and  implement  Pre-Planned  Responses  that  will  be  effective  mechanisms  to 

detect,  deter,  defend  and  mitigate  likely  terrorist  threats  against  U.S.  Navy  assets  and 

installations. 

Enabling  Objectives: 

2.3b  Use  the  SavageStudio  visual  mode  to  demonstrate  the  assessment,  warning  and 
threat  zones  as  used  in  the  anti-terrorism  program  concept  of  defense  in  depth. 
(Knowledge)  Perfonnance  data  for  this  objective  will  be  collected  using  the 
SavageStudio,  student  will  explain  or  address  the  3  zones  as  they  apply  to  the 
scenario. 

2.3c  Use  the  SavageStudio  in  visual  mode  to  demonstrate  the  application  of  the 
Assessment,  Warning  and  Threat  Zones  as  used  in  the  anti-terrorism  program. 
(Skill)  Performance  data  for  this  objective  will  be  collected  using  the 
SavageStudio,  student  will  demonstrate  or  identify  the  3  zones  as  they  apply  to 
the  scenario. 

2.5a  Use  the  SavageStudio  in  visual  mode  to  identify  critical  assets,  and  vulnerabilities 
for  the  NAVSTA  scenario.  (Knowledge)  Performance  data  for  this  objective  will 
be  collected  using  the  SavageStudio,  describe  and  explain  the  assessment  of  the 
environment  and  situation,  the  student  will  explain  potential  threats  and 
vulnerabilities  as  they  apply  to  the  scenario. 

2.5b,  Use  the  SavageStudio  in  visual  mode  to  explain  the  anti-terrorism  environment 
and  situation;  using  the  X3D  visual  scene  of  the  generated  scenario  explain  the 
force  protection  employment  of  assets  to  protect  given  critical  assets  while 
minimizing  vulnerabilities.  (Skill)  Perfonnance  data  for  this  objective  will  be 
collected  using  the  SavageStudio,  the  student  will  identify  potential  threats  and 
vulnerabilities  as  they  apply  to  the  scenario. 
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Task  2  SAT  UNSAT 


Set  up  Defense  in  Depth  for  the  NAVSTA  Scenario 

N/A 

N/A 

Assessment  zone  established  or  addressed? 

Warning  zone  established  or  addressed? 

Threat  zone  established  or  addressed? 

List  the  pre-planned  responses  for  boat-born  IED. 

Asses  the  environment  and  situation 

N/A 

N/A 

Identify  critical  assets 

Identify  vulnerabilities 

Exercise  scenario  to  describe  the  principles  of  defense  in  depth 
as  they  occur  in  the  scenario. 

Observe  the  pre-planned  responses  for  boat-born  IED. 
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APPENDIX  N.  ANALYST  REPORT 


A.  ANALYST  REPORT  INTERFACE 

The  following  figures  illustrate  the  Viskit  interface  used  for  creating  the  analyst  report.  A 
brief  description  is  provided  for  each  figure. 

1.  The  Heading  Panel  is  used  to  annotate  the  report  information.  Information 
required  is  title  of  the  report,  author’s  name,  date,  time  and  classification  of  the  report. 


Figure  38.  Viskit  Analyst  Report  Editor  displaying  the  Heading  Panel  window. 
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2.  The  Executive  Summary  allows  the  analyst  to  provide  a  synopsis  of  the 
simulation.  It  provides  the  context  for  the  remainder  of  the  report  and  ensures  that 
someone  reading  the  report  has  an  overview  of  the  entire  simulation  problem 


Figure  39.  SavageStudio  scenario  generation  display  window  depicting  scenario  loading 

from  the  scenario  run  library. 

3 .  Simulation  Location  Panel  is  used  to  provide  information  about  the  simulation 
environment.  The  comments  in  this  section  allow  before  and  after  simulation 
comparisons  from  the  analyst  point  of  view. 


Figure  40.  SavageStudio  scenario  generation  display  window  depicting  scenario  loading 

from  the  scenario  run  library. 
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4.  The  Simulation  Configuration  panel  is  used  to  enumerate  all  of  the  entities  and 
objects  employed  in  the  simulation.  A  table  of  SimEntities  is  auto-generated  from  the 
assembly  output  file  which  lists  the  description  of  the  entity  as  well  as  the  name  of  the 


Figure  41.  SavageStudio  scenario  generation  display  window  depicting  scenario  loading 

from  the  scenario  run  library. 


5.  The  Entity  Parameters  panel  allows  the  analyst  to  make  annotations  about  the 
parameters  used  in  the  simulation.  The  panel  provides  the  ability  to  preview  the 
parameters  that  will  be  incorporated  into  the  simulation.  Analysts  are  required  to  point 
out  parameters  that  were  used  to  influence  the  outcome  of  the  simulation. 


Figure  42.  SavageStudio  scenario  generation  display  window  depicting  scenario  loading 

from  the  scenario  run  library. 
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6.  The  Behavior  Definitions  panel  allows  before  and  after  annotations  regarding  the  level 
of  detail  about  behavior  definition  descriptions.  The  Analysts  are  required  to  provide  a 
description  about  the  behaviors  used  in  the  simulation. 


Figure  43.  SavageStudio  scenario  generation  display  window  depicting  scenario  loading 

from  the  scenario  run  library. 


7.  The  Statistical  Results  panel  allows  a  user  to  provide  before  and  after 
comments  about  the  replication  and  summary  statistics  for  a  specific  simulation.  The 
analyst  has  the  option  whether  or  not  he  will  like  to  include  histogram  charts  of  the 
replication  statistics  as  part  of  their  output. 


Figure  44.  SavageStudio  scenario  generation  display  window  depicting  scenario  loading 

from  the  scenario  run  library. 
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8.  The  Conclusions  and  recommendations  panel  allows  the  analysts  to  use  the 
interface  to  provide  comments  regarding  the  results  of  the  simulation  and 
recommendations  for  future  work. 


Figure  45.  SavageStudio  scenario  generation  display  window  depicting  scenario  loading 

from  the  scenario  run  library. 
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B. 


ANALYST  REPORT  EXAMPLE 


THIS  REPORT  IS:  UNCLASSIFIED 

AT/FP  training  analysis  for  the  Bremerton 

scenario 

Analyst:  Participant  number  20 
Analysis  date:  7/22/07  1:21  PM  3/19/07  2:17  PM 


Executive  Summary 

The  purpose  of  this  report  is  to  analyze  the  participants  force  protection  implementation 
and  security  assets  employment  given  a  specific  threat  as  identified  in  the  task  sheet.  The 
participant  reviews  the  generated  report  and  analyze  the  results  and  determines  mission 
success.  If  the  mission  is  not  successful  the  participant  returns  to  the  SavageStudio 
application  to  modify  asset  employment. 


Simulation  Location 

Description  of  Location  Features.  The  scenario  generated  by  this  simulation  will  be  of 
Bremerton  Harbor  by  participant  number  20. 


Post-Experiment  Analysis  of  Significant  Location  Features.  Participant  number  20 
completed  83%  of  training  objectives.  He  did  not  properly  assess  the  port  vulnerabilities. 
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Assembly  Configuration  for  Viskit  Simulation 

Assembly  Design  Considerations.  Assets  used  by  the  participant  included:  1  Nimitz 
Carrier,  2  explosive-laden  vessels,  2  Rhib  patrol  boats  and  Barriers 

Post-Experiment  Analysis  of  Simulation  Assembly  Design.  Barrier  employment  and  patrol 
areas  assignment  affect  the  explosive-laden  vessels  attack  profile. 
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Simulation  Entities 


Entity  Name 

Behavior  Definition 

PierBravo 

Obstacles.Pier 

PierCharlie 

Obstacles.Pier 

PierDelta 

Obstacles.Pier 

MooringE 

Obstacles.Pier 

MooringF 

Obstacles.Pier 

MooringG 

Obstacles.Pier 

MooringBuoyA  1 3 

Obstacles. Buoy 

MooringBuoyA  1 2 

Obstacles. Buoy 

MorringBuoyAl  1 

Obstacles. Buoy 

MooringBuoyL  1 

Obstacles. Buoy 

MooringBuoyL2 

Obstacles. Buoy 

MooringBuoyL3 

Obstacles. Buoy 

MooringBuoyL4 

Obstacles. Buoy 
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T  erroristCellPlanner 

Hostile.  T  erroristCellPlanner 

Terrorist  1 

Hostile. ExplosiveLadenVessel 

Terrorist2 

Hostile. ExplosiveLadenVessel 

CVN70 

Friendly.  MilitaryShip 

PierBravo 

Obstacles.Pier 

PierCharlie 

Obstacles. Pier 

PierDelta 

Obstacles.Pier 

MooringE 

Obstacles.Pier 

MooringF 

Obstacles.Pier 

MooringG 

Obstacles.Pier 

RHIB 

Friendly. MilitaryPatrolCraft 

NauticalChart 

Utilities.NauticalChart 

MooringBuoyA  1 3 

Obstacles. Buoy 

MooringBuoyA  1 2 

Obstacles. Buoy 

MorringBuoyAl  1 

Obstacles. Buoy 

MooringBuoyL  1 

Obstacles. Buoy 

MooringBuoyL2 

Obstacles. Buoy 

MooringBuoyL3 

Obstacles. Buoy 

MooringBuoyL4 

Obstacles. Buoy 

Entity  Parameters 

Entity  Parameters  Overview.  Entity  parameters  are  initialization  values  used  to  define 
new  event  graphs.  These  values  are  pulled  directly  from  the  assembly. 


Behavior  Definitions 

Description  of  Behavior  Design.  The  following  table  specifies  the  specific  parameters  for 
entities  and  objects  in  this  simulation. 

Post-Experiment  Analysis  of  Entity  Behaviors.  The  following  table  specifies  the  specific 
parameters  for  entities  and  objects  in  this  simulation. 


Statistical  Results 

Description  of  Expected  Results.  2  replications  were  evaluated  for  statistical  purposes. 

Analysis  of  Experimental  Results.  Comments  Participants  are  allowed  to  review  their 
statistical  results  and  encouraged  to  alter  the  scenario  with  different  configurations  and 
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evaluate  if  any  modification  to  their  original  scenario  makes  improvements.  The  main 
focus  of  the  experiment  is  to  observe  the  participants  taking  the  correct  steps  to  protect 
the  assets  assigned,  not  the  statistical  results.  Terrorists  can  successfully  attack  an  asset 
even  if  proper  employment  of  assets  is  observed. 

As  an  exemplar,  no  analysis  was  performed.  Other  potential  useful  statistics  would  be 
time  to  detect  all  targets  or  the  amount  of  detections  per  target. 
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Replication  Report 
Entity:  Rhib 

Property:  Rhib2Detections 

oji 

023 
031 
030 
0  10 
Oil 
0  17 
0  10 
0  10 
0  14 

Off 

0  13 
0  11 
0  10 
000 
000 
007 
000 
000 
004 
003 
003 
0  01 
000 

OO  Of  10  to  20  30  30  )9  40  4S  00  10  00  00  7i 

StadtootOolKlOM 

lSo»»*Qf€«tocoom| 


Run# 

Count 

Min 

Max 

Mean 

StdDev 

Variance 

1 

14.000 

0.000 

13.000 

6.500 

4.183 

17.500 

2 

4.000 

0.000 

3.000 

1.500 

1.291 

1.667 

3 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

4 

6.000 

0.000 

5.000 

2.500 

1.871 

3.500 

5 

6.000 

0.000 

5.000 

2.500 

1.871 

3.500 

6 

4.000 

0.000 

3.000 

1.500 

1.291 

1.667 

7 

5.000 

0.000 

4.000 

2.000 

1.581 

2.500 

8 

8.000 

0.000 

7.000 

3.500 

2.449 

6.000 

9 

4.000 

0.000 

3.000 

1.500 

1.291 

1.667 

10 

5.000 

0.000 

4.000 

2.000 

1.581 

2.500 

11 

4.000 

0.000 

3.000 

1.500 

1.291 

1.667 

12 

4.000 

0.000 

3.000 

1.500 

1.291 

1.667 

13 

14.000 

0.000 

13.000 

6.500 

4.183 

17.500 

14 

7.000 

0.000 

6.000 

3.000 

2.160 

4.667 
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15 

6.000 

0.000 

5.000 

2.500 

1.871 

3.500 

16 

16.000 

0.000 

15.000 

7.500 

4.761 

22.667 

17 

3.000 

0.000 

2.000 

1.000 

1.000 

1.000 

18 

5.000 

0.000 

4.000 

2.000 

1.581 

2.500 

19 

5.000 

0.000 

4.000 

2.000 

1.581 

2.500 

20 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

21 

18.000 

0.000 

17.000 

8.500 

5.339 

28.500 

22 

5.000 

0.000 

4.000 

2.000 

1.581 

2.500 

23 

8.000 

0.000 

7.000 

3.500 

2.449 

6.000 

24 

6.000 

0.000 

5.000 

2.500 

1.871 

3.500 

25 

5.000 

0.000 

4.000 

2.000 

1.581 

2.500 

26 

4.000 

0.000 

3.000 

1.500 

1.291 

1.667 

27 

5.000 

0.000 

4.000 

2.000 

1.581 

2.500 

28 

5.000 

0.000 

4.000 

2.000 

1.581 

2.500 

29 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

30 

1.000 

0.000 

0.000 

0.000 

0.000 

0.000 

31 

6.000 

0.000 

5.000 

2.500 

1.871 

3.500 

32 

6.000 

0.000 

5.000 

2.500 

1.871 

3.500 

33 

7.000 

0.000 

6.000 

3.000 

2.160 

4.667 

34 

7.000 

0.000 

6.000 

3.000 

2.160 

4.667 

35 

6.000 

0.000 

5.000 

2.500 

1.871 

3.500 

36 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

37 

5.000 

0.000 

4.000 

2.000 

1.581 

2.500 

38 

6.000 

0.000 

5.000 

2.500 

1.871 

3.500 

39 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

40 

7.000 

0.000 

6.000 

3.000 

2.160 

4.667 

41 

3.000 

0.000 

2.000 

1.000 

1.000 

1.000 

42 

6.000 

0.000 

5.000 

2.500 

1.871 

3.500 

43 

7.000 

0.000 

6.000 

3.000 

2.160 

4.667 

44 

6.000 

0.000 

5.000 

2.500 

1.871 

3.500 

45 

5.000 

0.000 

4.000 

2.000 

1.581 

2.500 

46 

5.000 

0.000 

4.000 

2.000 

1.581 

2.500 

47 

8.000 

0.000 

7.000 

3.500 

2.449 

6.000 

48 

6.000 

0.000 

5.000 

2.500 

1.871 

3.500 
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49 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

50 

14.000 

0.000 

13.000 

6.500 

4.183 

17.500 

Replication  Report 
Entity:  Rhib 

Property:  Rhib2Detections 
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Run# 

Count 

Min 

Max 

Mean 

StdDev 

Variance 

1 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

2 

12.000 

0.000 

11.000 

5.500 

3.606 

13.000 

3 

11.000 

0.000 

10.000 

5.000 

3.317 

11.000 

4 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

5 

8.000 

0.000 

7.000 

3.500 

2.449 

6.000 

6 

11.000 

0.000 

10.000 

5.000 

3.317 

11.000 

7 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

8 

11.000 

0.000 

10.000 

5.000 

3.317 

11.000 

9 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

10 

8.000 

0.000 

7.000 

3.500 

2.449 

6.000 

11 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

12 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 
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13 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

14 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

15 

11.000 

0.000 

10.000 

5.000 

3.317 

11.000 

16 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

17 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

18 

11.000 

0.000 

10.000 

5.000 

3.317 

11.000 

19 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

20 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

21 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

22 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

23 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

24 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

25 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

26 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

27 

8.000 

0.000 

7.000 

3.500 

2.449 

6.000 

28 

8.000 

0.000 

7.000 

3.500 

2.449 

6.000 

29 

11.000 

0.000 

10.000 

5.000 

3.317 

11.000 

30 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

31 

11.000 

0.000 

10.000 

5.000 

3.317 

11.000 

32 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

33 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

34 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

35 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

36 

11.000 

0.000 

10.000 

5.000 

3.317 

11.000 

37 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

38 

8.000 

0.000 

7.000 

3.500 

2.449 

6.000 

39 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

40 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

41 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

42 

11.000 

0.000 

10.000 

5.000 

3.317 

11.000 

43 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

44 

8.000 

0.000 

7.000 

3.500 

2.449 

6.000 

45 

10.000 

0.000 

9.000 

4.500 

3.028 

9.167 

46 

11.000 

0.000 

10.000 

5.000 

3.317 

11.000 
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47 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

48 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

49 

9.000 

0.000 

8.000 

4.000 

2.739 

7.500 

50 

8.000 

0.000 

7.000 

3.500 

2.449 

6.000 

Summary  Report 


Entity 

Property 

Count 

Min 

Max 

Mean 

StdDev 

Variance 

Rhib 

Rhib  1  Detections 

2.000 

0.000 

1.000 

0.500 

0.707 

0.500 

Rhib 

Rhib2Detections 

2.000 

0.000 

11.294 

5.647 

7.986 

63.782 

Conclusions  and  Recommendations 

Conclusions.  Participant  number  20  completed  the  simulation  assessment  portion  with 
83%  success  rate. 

Recommendations  for  Future  Work  Based  on  the  results  the  subject  matter  expert 
recommends  that  participant  number  20  reviews  the  following  subjects:  how  to  identify 
vulnerabilities  and  threat  assessment  portion  of  the  lesson. 


This  report  was  autogenerated  by  Viskit  modeling  tool  using  the  Simkit  discrete- 
event  simulation  (DES)  libraries,  online  at  https ://diana.nps.edu/Simkit 
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APPENDIX  O.  SUPPORTING  MATERIALS 


A.  AT/FP  LESSON  DVD  CONTENTS 

The  AT/FP  lesson  DVD  is  organized  as  follows: 

CBTAndSimulation  lesson  folder 
Data  folder 
Narration  folder 
Player  folder 
Lesson  player 
CBT  VS  CI  lesson  folder 
Data  folder 
Narration  folder 
Player  folder 
Lesson  player 

B.  AT/FP  LESSON  DVD  REQUESTS 

The  DVD  that  contains  the  CBT  lessons  created  for  this  thesis  can  be  requested 
via  mail  from  the  Savage  Research  Group. 

Naval  Postgraduate  School,  MOVES  Institute,  Monterey,  California. 

Terry  Norbraten  or  Don  Brutzman 
1  University  Circle,  Monterey  Ca.  93943. 
e-mail  requests  can  be  made  to: 

Terry  Norbraten  tdnorbra@nps.edu 
Don  Brutzman  Brutzman@nps.edu 
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